# Prepending Input layer to pre-trained model

I'm trying to input numpy arrays of shape (1036800,) - originally images of shape (480, 720, 3) - into a pre-trained VGG16 model to predict continuous values.

I've tried several variations of the code below:

input = Input(shape=(1036800,), name='image_input')
initial_model = VGG16(weights='imagenet', include_top=False)
x = Flatten()(initial_model(input).output)
x = Dense(200, activation='relu')(x)
x = Dense(1)(x)
model = Model(inputs=input, outputs=x)


Previous variations of the above code yielded errors related to the input being the wrong dimensions, input_shape needing to have 3 channels (when using (1036800,) for that parameter in the initialization of VGG16), and the most recent error that results from running the above code is this:

Traceback (most recent call last):
File "model_alex.py", line 57, in <module>
model = initialize_model()
File "model_alex.py", line 20, in initialize_model
x = Flatten()(initial_model(input).output)
File "/home/aicg2/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 596, in __call__
output = self.call(inputs, **kwargs)
File "/home/aicg2/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 2061, in call
output_tensors, _, _ = self.run_internal_graph(inputs, masks)
File "/home/aicg2/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 2212, in run_internal_graph
output_tensors = _to_list(layer.call(computed_tensor, **kwargs))
File "/home/aicg2/.local/lib/python2.7/site-packages/keras/layers/convolutional.py", line 164, in call
dilation_rate=self.dilation_rate)
File "/home/aicg2/.local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 3156, in conv2d
data_format='NHWC')
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/nn_ops.py", line 639, in convolution
input_channels_dim = input.get_shape()[num_spatial_dims + 1]
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 500, in __getitem__
return self._dims[key]
IndexError: list index out of range


Here is the full code. Here is the sample data file used in the script.

One of the possible approach towards fixing this might be to resize the raw image files to 224x224 and turn them into numpy arrays of shape (224, 224, 3) so they can be plugged into the pre-trained model's first layer. However, I don't want to warp the images or waste another night pre-processing data when I should already be training.

Besides that, I all I can think to do is Google my problem and try to adapt the found solutions or aimlessly tweak various shape related parameters and functions -- neither of which has gotten me very far over the past 4 hours.

The issue is that you shouldn't flatten the images into 1-dimensional vector because the VGG16 contains 2D convolution layers (e.g. spatial convolution over images), which require the input to have the shape of (number_of_images, image_height, image_width, image_channels), given that keras.backend.image_data_format() returns 'channels_last'. If your image_data_format is 'channels_first', change the input data shape to (number_of_images, image_channels, image_height, image_width).

Here is your fixed code (tested with Keras 2.0.4):

x_train = x_train.reshape((x_train.shape[0], 480, 720, 3))
x_test = x_test.reshape((x_test.shape[0], 480, 720, 3))

initial_model = VGG16(weights='imagenet', include_top=False)
input = Input(shape=(480, 720, 3), name='image_input')
x = Flatten()(initial_model(input))
x = Dense(200, activation='relu')(x)
x = BatchNormalization()(x)
x = Dropout(0.5)(x)
x = Dense(1)(x)
model = Model(inputs=input, outputs=x)

model.fit(x_train, y_train, epochs=20, batch_size=16)
score = model.evaluate(x_test, y_test, batch_size=16)

• Excellent -- I appreciate the detailed explanation with the fixed code included. – aweeeezy Jul 22 '17 at 17:49