2
$\begingroup$

Should BatchNormalization be used only in CNNs or can they be used in Fully Connected Networks, Recurrent networks as well?

$\endgroup$
3
$\begingroup$

A Batch Normalization layer essentially is normalizing outputs of your hidden units, so their outputs would always have a similar scale. By eliminating such internal covariate shift in outputs of hidden layers, your deeper layers become more independent from their previous layers.

Nothing here is CNN specific - Batch Normalization may be applied to Fully Connected and Recurrent neural networks as well, but they are more useful with deep neural networks, which tend to accumulate this shift with each layer activated during Forward Propagation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.