I recently learned concepts of transfer learning. Is it necessarily true that fine-tuning of transferred layers perform better than frozen transferred layer? why?
Transfer learning means to apply the knowledge that some machine learning model holds (represented by its learned parameters) to a new (but in some way related) task. Whereas fine-tuning means taking some machine learning model that has already learned something before (i.e. been trained on some data) and then training that model (i.e. training it some more, possibly on different data).
From this we can conclude that if we are to use the learning of one model and concentrate only on some specific part of it, we can use transfer learning and train the network again with only a little amount of data. Whereas if we have sufficient data that we want the model to be trained upon, fine-tuning the model will come to the rescue. Refer to the previous question and this article
It depends on the number of data you have. If you have enough data you can train the entire network, if you don't have, the other one is better. The reason is somehow clear and is due to the nature of the convolutional networks. The first layers learn simple lines and edges and such primitive structures. The next layers try to somehow connect the previous patterns. If you have so much data, you can ignore the pretrained model by updating the pretrained weights. If you don't have enough data, you can freeze the first layers due to the mentioned reason. Deep layers, more specifically, the fully connected layers are for classification. You should let them be trained even if you don't have much data.