I'm using CIFAR 10 dataset, and I'm applying CNN.

I'm confused about my validation/test loss/accuracy curves and their behavior when I apply batch normalization and/or dropout. They fluctuate significantly when I add batch normalization and dropout, which I guess is expected, since Keras drops them for validation/test sets. It seems to me, however, that dropout alone does better job than with batch normalization. Is it something expected? Should I still use batch normalization in below example? (Also, I seem to be stuck with under 80% accuracy, what would be the best way to improve this model?)

My architecture is next (I set adam optimizer, learning rate 0.0001):

architecture pic

Result for this model is next:

Full model

I'm concerned about fluctuation in test loss/accuracy.

Model without dropout and batch normalization clearly suffers:

without dropout and bn

Model without dropout seems better, but test loss is significantly higher than training loss:

without bn

Model without batch normalization:

enter image description here

Model without batch normalization and default learning rate:

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.