0
$\begingroup$

I recently learned concepts of transfer learning. Is it necessarily true that fine-tuning of transferred layers perform better than frozen transferred layer? why?

$\endgroup$
  • $\begingroup$ what do you mean by frozen transferred layer? $\endgroup$ – thanatoz Nov 14 '18 at 19:54
  • $\begingroup$ where the weights are directly transferred from the pre-trained model and no further training or updating of weights is done $\endgroup$ – Adex Nov 14 '18 at 20:02
1
$\begingroup$

Transfer learning means to apply the knowledge that some machine learning model holds (represented by its learned parameters) to a new (but in some way related) task. Whereas fine-tuning means taking some machine learning model that has already learned something before (i.e. been trained on some data) and then training that model (i.e. training it some more, possibly on different data).

From this we can conclude that if we are to use the learning of one model and concentrate only on some specific part of it, we can use transfer learning and train the network again with only a little amount of data. Whereas if we have sufficient data that we want the model to be trained upon, fine-tuning the model will come to the rescue. Refer to the previous question and this article

| improve this answer | |
$\endgroup$
0
$\begingroup$

It depends on the number of data you have. If you have enough data you can train the entire network, if you don't have, the other one is better. The reason is somehow clear and is due to the nature of the convolutional networks. The first layers learn simple lines and edges and such primitive structures. The next layers try to somehow connect the previous patterns. If you have so much data, you can ignore the pretrained model by updating the pretrained weights. If you don't have enough data, you can freeze the first layers due to the mentioned reason. Deep layers, more specifically, the fully connected layers are for classification. You should let them be trained even if you don't have much data.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.