Transfer learning: Take a trained neural network and use it for a new classification task.

When we want to use transfer learning with a convolutional neural network, we don't have to use the same image size as input than the image size used for training. But if we change the input size we will have to re-train fully connected layers. See this post on Stackoverflow.

I don't undertand why changing input shape will not affect convolutional layer weight and why it will affect fully connected layer weights.

Tell me if my question is not understanble.

  • $\begingroup$ Can you add some references regarding your claim? Is it your own experience or are there some other resources which support this observation? $\endgroup$
    – Peter
    Aug 16, 2019 at 11:21
  • $\begingroup$ @Peter look at accepted answer here: stackoverflow.com/questions/41907598/… $\endgroup$
    – akhetos
    Aug 16, 2019 at 12:34
  • $\begingroup$ May be i'm not fully understanding the answer $\endgroup$
    – akhetos
    Aug 16, 2019 at 12:35
  • $\begingroup$ What is your question ? and more important how is it substantially different from your linked to question ? $\endgroup$ Aug 16, 2019 at 17:49
  • $\begingroup$ @ScottStensland Why transfer learning with a CNN with different input shape (than shape used for originial training ) involve training again fully connected layer? $\endgroup$
    – akhetos
    Aug 16, 2019 at 20:54

2 Answers 2


To understand this you need some basic mathematics of how matrix operations work. So let us start, consider you are using resnet-34 architecture which is trained on imagenet with 1000 classes so when you use tranfer learning you load the model architecture and its weights a model like resnet34 has two things backbone that is the convolution part and FCL's the neck of the network now when you train a model using resnet 34 it will only train the neck part now let us understand, convolutions work on whole images they are filters of some nxn dimension which work on a image of suppose dxk dimension now convolution filter sizes remain the same what changes is the feature map size now if you see usisg a nxn filter on dxk image will give you a feature map of (d-n+1)x(k-n+1) dimension for that particular filter now for the other convolution units they will perform convolutions in the same way producing these feature maps and different image size wont affect how convolutions take place thus this answers why input shape doesn't require weight changes for convolution but if image shape that has been changed and when convolotions are performed on them using tranfer learning models which are trained on different image sizes(Imagenet data) , then you will see that just before fcl's there is a flattening operation which flattens the final feature maps which then act as input for the fcl's now just see imagenet model will produce diffrent size feature maps which when flattened will have diffrent dimension as compared to yours when it will be flattened thus total number of input neurons for the fcl's may get reduced or increased depending upon your image size(feature map size after convolution +pooling) and when you reduce the number of neurons you will need to retrain them to get new weight values and also your output layer is also modified depending upon total number of classes you are predicting hence this ANN that is the fcl's there architecture is modified as per your image size and output classes hence changing the architecture requires changing the weights which is done using training through backpropogation.Thus this concludes image sizes and your problem dont affect the convolution(because weights of filters are updated and image size doesn't affect filter size but only the size of the feature map produced after convolution) process but they do affect the ANN or the fcl's hence weights are updated through training.

P.S- you can actually even update the weights of the convolutions by unfreezing the whole model and then re-training the whole network even the backbone fastai provides a great support for this visit there documentation and site it is quite helpful.

  • $\begingroup$ thanks for the clear answer! Just a last question, when we says CNN update convolutional weight, it's only update value of of each filter of size n*n right? or there is also a weight associated to each filter? $\endgroup$
    – akhetos
    Aug 21, 2019 at 7:48
  • 1
    $\begingroup$ Weights of filters are updated because a filter is multiplied by sliding over the pixel Matrix which results in the formation of a feature map .a nxn filter has nsqaure weights thus the total number of entries in your filter or "kernel" is equal to the total number of weights if you give it some time you will get the intution behind it. $\endgroup$ Aug 21, 2019 at 8:04
  • 1
    $\begingroup$ Thanks for the answer! But please, please start using punctuation; it's really hard to read like this! $\endgroup$
    – karu
    Oct 4, 2022 at 8:33

In convolution layers, weights are the filter values.
Weights of a CNN network is determined by the number and size of filters. Weights doesn't depend on shape of input image.
Filters of pre-trained network are capable of detecting different features in an image. Consider sobel filter which detects edges in an image. A 3x3 sobel filter can be used to detect edges in an image of any size. Similarly, those filters can be reused instead of again learning from scratch.

That's why, weights of CNN network can be transferred to another CNN network with different input shape. However, number of layers, no. of filters, filter sizes and number of image channels must be same.

For transfer learning, best practices would be to use pre-trained model for similar task and don't change the input shape to very small or large.

On the other hand, weights of Fully Connected(Dense) layer can't be transferred. Because, those weights depend on image size. Dense layers at the end of CNN network should be cut off and recreated to adopt to change in image size.
It would be more clear if you get to know how the trainable parameters are calculated in CNN and Dense layers. It would be very long if I try to explain in this answer. You can google search on that.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.