So from what i understand, transfer learning is the fact of training a model on a dataset where you have a lot of data, then keeping most of trained coefficients, and only re-training the last layer on a dataset for a problem where you have a little amount of data

And feature extraction for CNN means that you have a generic model which extracts features from the image, and then you only need to train the last layer to solve a particular problem on images.

So is feature extraction just a particular case, or is there some subtlety i did not understand ?


1 Answer 1


Transfer learning is, as you said, retraining the final layer of a deep network. Not only is this useful for solving problems with limited training examples, but also when you don't have adequate computing resources to train a network from scratch. Some models have hundreds of millions of parameters, which could take weeks to train if you only have modest equipment. However, if you have sufficient data, adapting weights via transfer learning is not preferable because the features that were extracted from the original training process are unlikely to be ideal for another application.

Feature extraction, in the context of CNNs, is not necessarily an explicit process, rather a sort of high-level product of the training process. Feature extraction refers to the portion of the training process by which a CNN learns to map input space to a latent space that can subsequently be used for classification via the final layer. In other words, the hidden layers learn discriminatory features in the form of weight-adjusted (usually by backpropagating the error) convolutional filters. Thus the term "feature extraction" generally refers to the portion of the training process that occurs before the final layer. So it is not part of transfer learning in which only the last layer is trained.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.