Scenario: I've been training a CNN for the cifar10 dataset. I'm using tensorflow, and a CNN with 12 conv layers and 1 dense layer before a softmax dense layer. I'm using data augmentation as well with batch normalization.
After a few hundred epochs I archieved a maximum of 92.73 percent accuracy on the validation set.
- Validation loss goes up slightly as I train more.
- While validation loss goes up, validation accuracy also goes up.
- One epoch gave me a loss of 0.295, with a validation accuracy of 90.5%. My best epoch for validation accuracy gave me 92.73% with a validation loss of 0.33.
- Why is my validation accuracy increasing while my validation loss is going up?
- Should I use a loss metric diferent to cross_entropy?