Which batch size to use when Batch Normalization?

Question

I want to train a CNN in Keras (optimizer Adam) and by using batch normalization after every ConvLayer and before every activation layer. So far I mostly see examples in which training is carried out with a batch size of 32 or 64 samples. Should it not be ensured that the last batch in every epoch still contains 32 or 64 samples and not significantly less samples? What do I mean, if I have 500 training samples for training, would a batch size of 50 not be better than a batch size with 32 oder 64 samples?

Answer 1

The fact that training size is not divisible by batch size does not matter (primarily)

main reasons for batch training is it requires less memory. Since you train the network using fewer samples, the overall training procedure requires less memory, also speed. Typically networks train faster with mini-batches. That's because we update the weights after each propagation.

What you have left in the last iteration will just be propagated even though its not divisible.