How to use efficient net as feature extractor for meta/Few shot learning in PyTorch

Question

I am working on few shot learning and I wanted to use efficient-net as backbone feature extractor. Most of the model use Resnet as feature extractor. For example I can use below line of code and it extract features for me -

from model.res50 import ResNet
self.encoder = ResNet()
self.fc = nn.Linear(hidden_dim, num_classes)
def forward(self, data):
    out = self.encoder(data)
    out = self.fc(out)
    return out 

I am using this PyTorch implemetation of Efficientnet- EfficientNet-PyTorch. I am not sure how to use this effcientnet as feature extractor as I did with Resnet. Help is highly appreciated.

Answer 1

Just to make things clear, features of a model are raw model outputs before the fully connected layers, i.e. output of your backbone. So in your code, you get the features of the model as shown in comments -

from model.res50 import ResNet
self.encoder = ResNet()
self.fc = nn.Linear(hidden_dim, num_classes)
def forward(self, data):
    out = self.encoder(data) # <-- This line gives you the features of model (backbone) in the 'out' tensor
    out = self.fc(out) # <-- These features are then fed to fully connected layers to produce final result. At the time of final predcition, activation functions are used (eg sigmoid or softmax).
    return out 

These features of a CNN can be of shape 1D, 2D or 3D depending on the architecture. So in the link that you have mentioned, you can get the features of efficientnet using -

x = self.extract_features(inputs) # refer line number 314 in model.py

Then global average pooling is done to further reduce the size of output tensor (originally done by the authors of Resnet). So it depends on you whether you want to do global average pooling or not. And finally these features are flatten (resized into 1D shape). Ideally I would use this flatten tensor as the model features to perform few-shot learning.

That is how you can get the features. And we don't consider the output of fully connected layers without activation are features.