0
$\begingroup$

I'm using a ResNet50 model pretrained on ImageNet, to do transfer learning, fitting an image classification task.

The easy way of doing this is simply freezing the conv layers (or really all layers except the final fully connected layer), however I came across a paper where the authors mention that batch normalisation layers should be fine tuned when fitting the new model:

Few layers such as Batch Normalization (BN) layers shouldn’t be froze because, the mean and variance of the dataset will be hardly matching the mean or variance from pre-trained weights. So, auto-tuning [30] is adapted for the BN layers in ResNey50, i.e. few of the layers which are in the top of ResNet50 shouldn’t be frozen.

source here

I initially froze everything, but in the interest of time and not having to commit computation to testing the effect of not freezing the BN layers, how important is it to not freeze them?

$\endgroup$

1 Answer 1

0
$\begingroup$

The batch normalization layer helps with effectively training the model. Since you are transfer learning, you may have frozen everything up to the fully connected classifier. If you want to keep the parameters of the frozen layers exactly the same as the original model, you can load the weights of only the retrained head during inference and/or evaluation.

Your best bet is to try everything and see what works best for you.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.