1
$\begingroup$

It is common practice to use the standard scaler on the inputs before feeding it to a deep learning architecture. I was wondering whether it is necessary if the first layer is a batch normalization layer.

$\endgroup$

1 Answer 1

2
$\begingroup$

Scaling is a bit different from what Batch normalization does. Performing scaling creates scale indifference amongst all the data points. Ex: Values 5 and 55 will have a higher magnitude of scale difference than the log(5)=0.698 and log(55)=1.740. This is the idea behind scaling. Similarly, We scale the images with 255 which helps in faster convergence

When we pass in the scaled data to our network, due to the operations that take place in every layer of the network, the distribution of data observed in let's say 5th layer of the network is different from the distribution in the 1st layer. This is a problem because we are kind of training the first part of the network in one form of data and the latter part of the network with another form of data. So to remove this effect, We do BatchNorm in later layers of the network

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.