1
$\begingroup$

Transfer learning: Take a trained neural network and use it for a new classification task.

When we want to use transfer learning with a convolutional neural network, we don't have to use the same image size as input than the image size used for training. But if we change the input size we will have to re-train fully connected layers. See this post on Stackoverflow.

I don't undertand why changing input shape will not affect convolutional layer weight and why it will affect fully connected layer weights.

Tell me if my question is not understanble.

$\endgroup$
  • $\begingroup$ Can you add some references regarding your claim? Is it your own experience or are there some other resources which support this observation? $\endgroup$ – Peter Aug 16 '19 at 11:21
  • $\begingroup$ @Peter look at accepted answer here: stackoverflow.com/questions/41907598/… $\endgroup$ – akhetos Aug 16 '19 at 12:34
  • $\begingroup$ May be i'm not fully understanding the answer $\endgroup$ – akhetos Aug 16 '19 at 12:35
  • $\begingroup$ What is your question ? and more important how is it substantially different from your linked to question ? $\endgroup$ – Scott Stensland Aug 16 '19 at 17:49
  • $\begingroup$ @ScottStensland Why transfer learning with a CNN with different input shape (than shape used for originial training ) involve training again fully connected layer? $\endgroup$ – akhetos Aug 16 '19 at 20:54
1
$\begingroup$

To understand this you need some basic mathematics of how matrix operations work. So let us start, consider you are using resnet-34 architecture which is trained on imagenet with 1000 classes so when you use tranfer learning you load the model architecture and its weights a model like resnet34 has two things backbone that is the convolution part and FCL's the neck of the network now when you train a model using resnet 34 it will only train the neck part now let us understand, convolutions work on whole images they are filters of some nxn dimension which work on a image of suppose dxk dimension now convolution filter sizes remain the same what changes is the feature map size now if you see usisg a nxn filter on dxk image will give you a feature map of (d-n+1)x(k-n+1) dimension for that particular filter now for the other convolution units they will perform convolutions in the same way producing these feature maps and different image size wont affect how convolutions take place thus this answers why input shape doesn't require weight changes for convolution but if image shape that has been changed and when convolotions are performed on them using tranfer learning models which are trained on different image sizes(Imagenet data) , then you will see that just before fcl's there is a flattening operation which flattens the final feature maps which then act as input for the fcl's now just see imagenet model will produce diffrent size feature maps which when flattened will have diffrent dimension as compared to yours when it will be flattened thus total number of input neurons for the fcl's may get reduced or increased depending upon your image size(feature map size after convolution +pooling) and when you reduce the number of neurons you will need to retrain them to get new weight values and also your output layer is also modified depending upon total number of classes you are predicting hence this ANN that is the fcl's there architecture is modified as per your image size and output classes hence changing the architecture requires changing the weights which is done using training through backpropogation.Thus this concludes image sizes and your problem dont affect the convolution(because weights of filters are updated and image size doesn't affect filter size but only the size of the feature map produced after convolution) process but they do affect the ANN or the fcl's hence weights are updated through training.

P.S- you can actually even update the weights of the convolutions by unfreezing the whole model and then re-training the whole network even the backbone fastai provides a great support for this visit there documentation and site it is quite helpful.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ thanks for the clear answer! Just a last question, when we says CNN update convolutional weight, it's only update value of of each filter of size n*n right? or there is also a weight associated to each filter? $\endgroup$ – akhetos Aug 21 '19 at 7:48
  • 1
    $\begingroup$ Weights of filters are updated because a filter is multiplied by sliding over the pixel Matrix which results in the formation of a feature map .a nxn filter has nsqaure weights thus the total number of entries in your filter or "kernel" is equal to the total number of weights if you give it some time you will get the intution behind it. $\endgroup$ – khwaja wisal Aug 21 '19 at 8:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.