2
$\begingroup$

I build my CNN on Keras, normally in the ImageDataGenerator I saw the rescale = 1. / 255 used to normalize input data (pixel value) from [0-255] to [0-1]. Then I read about Batch Normalization Layer, I wonder if they are mutually-exclusive or can they be used together in the same network?

I have tried to adapted the BN Layer to my network (BN after every activation Layer) but the loss fluctuates more than without using BN.

$\endgroup$
  • 1
    $\begingroup$ since I haven't found any example regarding this so I gave it a try. As I understand the Batch normalization will normalize the input of each layer per batch, I'm not so clear about what if I have already normalized the whole input by using rescale $\endgroup$ – Thanh Nguyen Aug 1 '18 at 4:37
2
$\begingroup$
  1. Its is basically not really important to rescale your input to [0,1]. Your input data should simply be in the same range. So [0,255] would be also a legit range.

  2. BN should be different to interpret. It helps to keep your activations through the whole network on the same level. It is especially helpfull when we talk about deeper archtitectures. Actually when you spend some extra time/iterations, your network should also converge without BN.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.