# Batch normalization vs batch size

I have noticed that my performance of VGG 16 network gets better if I increase the batch size from $$64$$ to $$256$$. I have also observed that, using batch size $$64$$, the with and without batch normalization results have lot of difference. With batch norm results being poorer. As I increase the batch size the performance of with and without batch normalization gets closer. Something funky going on here.

So I would like to ask is the increase in batch size have effect on batch normalization?

• Adding batchnorm right after the relu layers? Nov 30 '18 at 6:57
• tried both after relu and before relu, both results are poorer than without BN. Nov 30 '18 at 15:39
• In Pytorch, for their faster R-CNN Resnet 50 fpn architecture in torchvision, I've seen it actually recommended to disable batch normalization since the images are 800 by 800 by 3 tensors, and even a batch of 8 with 16 bit floating precision will be too large for a lot of entry level ML GPUs. Feb 4 at 6:58