I was researching about "why are we freezing layers" and I came across the answer says "to not lose the information of pre-trained model" But; we are just freezing early layers (I know why). For example: our data is so similar to the data that the model trained on. Let's say we are not freezing any layer. The model will make very small mistakes and convergence will be less, we will not be destroying any information (even if, weights will change very little). Am I wrong? If I am not, then why are we freezing any layer?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ I feel like this may help you what-are-the-consequences-of-not-freezing-layers-in-transfer-learning $\endgroup$– Adrian BCommented Aug 30, 2022 at 9:04
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
If the data is already similar, it doesn't make sense to train the lower layers (backbone), as your network will already be good for extracting features. Then you freeze them to quickly train your classifier (head).
As stated in the link quoted by Adrian, new layers have large gradients in the first epocs and this can affect the model. So if your data is similar but with new information, large gradient updates during training will destroy your pre-trained features, It's applied for fine-tuning too and you can check here. https://keras.io/guides/transfer_learning/
If the data is different for example you want to train a pre-trained model on imagenet to classify brain tumors, then losing these features doesn't make much difference, it would be better to freeze only the first layers that already can extract low-level features such as the horizontal/vertical edges.
