# Why is input preprocessing in VGG16 in Keras not 1/255.0

I am just trying to use pre-trained vgg16 to make prediction in Keras like this.

from scipy import ndimage
from keras.preprocessing import image
from keras.applications.vgg16 import preprocess_input

im = scipy.misc.imresize(im, (224, 224)).astype(np.float32)

#im /= 255.0
#im = im - np.mean(im, axis=2, keepdims=True)

im = np.expand_dims(im, axis=0)
im = preprocess_input(im)

out = vgg16_model.predict(im)
np.argmax(out)


It seemed that im /= 255.0 give very bad prediction. I commented it out and it started making good prediction. I also added preprocess_input(...) but that doesn't seem to affect prediction for the few random trials I did.

The question is that according to this great blog:

https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

under "Using the bottleneck features of a pre-trained network: 90% accuracy in a minute", pre-trained VGG16 is in a transfer learning context. And if you look at this gist, you see this line of code:

def save_bottlebeck_features():
datagen = ImageDataGenerator(rescale=1. / 255)
etc.


The preprocessing of input seemed to be 1/255.0 during caching of features from the last conv layer. This is sort of puzzling. I further also looked up how preprocess_input(...) is defined in the code I have, and found for 'tf':

x /= 127.5
x -= 1.


Which you can check here.

• I would like to add a word of caution. If you play around with this code, be aware that preprocess_input actually modify its input inplace since it is passed by reference. preprocess_input doesn't attempt to make a copy before manipulating the image. – kawingkelvin Apr 21 '18 at 18:28
• Did you ever find a good anser as to why the input is scaled to -1 to 1 when using vgg16 on keras with tensorflow backend? – user3731622 Oct 17 '18 at 21:00
• @user3731622, see my comment below. This bit of detail doesnt really matter, in the context of using pre-trained weights and transfer learning, as long as your preprocessing is sensible (i.e. bounded in O(1) and with mean O(1).) In my case, other aspect of training matter much more in getting good results. – kawingkelvin Nov 12 '18 at 4:40

I have a new thinking on this. I think it maybe ok to use a different but reasonable preprocessing (such as 1/255.) in the context of transfer learning (pre-training), than whats originally used to train the VGG16. As long as the top most conv-conv-...pool representation is useful for your new task, and it empirically checked out fine for what F. Chollet did in his blog.

The pre-trained weights that are available on Keras are trained with the preprocessing steps defined in preprocess_input() function that is made available for each network architecture (VGG16, InceptionV3, etc).

For example

from keras.applications.vgg16 import preprocess_input

If you are using the weights that comes with keras for fine tuning, then you should use the corresponding preprocess_input() function for the network. It can be different from the original preprocessing steps mentioned in the paper.

from keras.applications.vgg16 import VGG16
model = VGG16(weights='imagenet', include_top=False)


If you want to stick to the original preprocessing steps, you can find pre-trained weights that is trained with the original preprocessing steps instead of using the weights that comes with Keras.

Maybe at the time of writing the blog post the weights were trained with different preprocessing steps.