feature extraction for a pretrained model in keras

Question

Keras has a way to extract the features of a pretrained model, described here https://keras.io/applications/

from keras.applications.vgg16 import VGG16
from keras.preprocessing import image
from keras.applications.vgg16 import preprocess_input
import numpy as np

model = VGG16(weights='imagenet', include_top=False)

img_path = 'elephant.jpg'
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

features = model.predict(x)

I tried this with some sample images. My vector x is of shape (100, 3, 224, 224) for 100 observations, 3 for RGB and 224x224 pixel size. the preprocess_input reshapes this for the VGG model (it expects a different order).

However, the output shape of features is (100, 512, 7, 7). What is this shape? I want to use the features as input for a logistic regression. So I need a shape like (100, n): one row for each observation and the features in the columns. How do I reshape the output to this dimension?

Say I now want to build my own simple Convnet:

from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense

model = Sequential()
model.add(Convolution2D(32, 3, 3, input_shape=(1, 299, 299)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Convolution2D(32, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Convolution2D(64, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

This model expects grayscale images as input, hence the shape.

What kind of layer do I have to add to get features of this model (something I can input in a logistic regression or random forest).

Thanks

Answer 1

The features variable contains the outputs of the final convolutional layers of your network. The final convolutional layer of VGG16 outputs 512 7x7 feature maps. All you need to do in order to use these features in a logistic regression model (or any other model) is reshape it to a 2D tensor, as you say.

reshaped_features = features.reshape(100, 512*7*7)

This will flatten out the feature maps to a long one-dimensional vector for each instance.

Tip: If you can't be bothered working out the actual dimensions for your reshape, you can replace 512*7*7 with -1 and numpy will figure out how big the final dimension should be.