Batch Normalization is a layer that is put in between convolution and activation layers or sometimes after activation layers. It is used to normalize layer’s input to reduce the internal covariate shift problem.
This problem occurs by changing in distribution of the input data in the early layers, and because every layer depends on the input of the former layers, it becomes a problem for this layer since it requires repeatedly adjusting to new input distributions.
Batch Normalization is a technique for training very deep neural networks that standardizes the inputs to a layer for each mini-batch. This has the effect of stabilizing the learning process and dramatically reducing the number of training epochs required to train deep networks.
It can be implemented during training by calculating the mean and standard deviation of each input variable to a layer per mini-batch and using these statistics to perform the standardization.
You read more here, here or here.
Also, I can tell you that this layer has a good impact on the performance and the achieved scores of your model.
Hope this helps you understand the importance of BN.
