The disadvantage of using transfer learning is that it cannot be layered to reduce the number of parameters. In that statement what are the layers of transfer learning and the number of parameters?
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
If you are looking for pre-trained model for your solution, you must already have a dataset which needs to be trained. However to get world class model, you need to build very deep networks, for e.g. Resnet50 has 50 layers, Resnet101 has 101 layers. Think of building those from scratch. It takes ages to train them.
So advantage of pre-trained model is you take existing model with already fitted weights and attach your last layer for new targets. Now, no. of trainable parameters should not be as large as original model. You need set layers unattainable with following code
for layer in resnet50.layers:
layer.trainable = False
This makes only the last layer trainable. Hope this helps
-
-
$\begingroup$ No that is Keras, using TF in the backend $\endgroup$ Commented Aug 29, 2020 at 16:54
$\begingroup$
$\endgroup$
Transfering learning is nothing but using the layers of a highly trained model in feature extraction of other models. Inception, MobileNet, VGG are some of the examples.
The disadvantage of using transfer learning is that it cannot be layered to reduce the number of parameters.
Note that we can only remove the last layer from these trained models. If we try to alter the intermediate layers, the model will lose its generalisation and finally will be of no use. The weights of a layer are dependent on the previous layers activations ( during backpropogation ) hence, we cannot alter the intermediate layers.
Now, in order to reduce the number of trainable parameters, maybe for low CPU consumption, we can reduce the number of layers in a model.
But for a pretrained model, this could not be possible as we cannot change the intermediate layers to reduce the trainable parameters.
