0
$\begingroup$

I'm using CIFAR 10 dataset, and I'm applying CNN.

I'm confused about my validation/test loss/accuracy curves and their behavior when I apply batch normalization and/or dropout. They fluctuate significantly when I add batch normalization and dropout, which I guess is expected, since Keras drops them for validation/test sets. It seems to me, however, that dropout alone does better job than with batch normalization. Is it something expected? Should I still use batch normalization in below example? (Also, I seem to be stuck with under 80% accuracy, what would be the best way to improve this model?)

My architecture is next (I set adam optimizer, learning rate 0.0001):

architecture pic

Result for this model is next:

Full model

I'm concerned about fluctuation in test loss/accuracy.

Model without dropout and batch normalization clearly suffers:

without dropout and bn

Model without dropout seems better, but test loss is significantly higher than training loss:

without bn

Model without batch normalization:

enter image description here

Model without batch normalization and default learning rate:

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.