I am just trying to use pre-trained vgg16 to make prediction in Keras like this.

from scipy import ndimage
from keras.preprocessing import image
from keras.applications.vgg16 import preprocess_input

im = scipy.misc.imread("cat_dog/validation/cats/cat.1362.jpg").astype(np.float32)
im = scipy.misc.imresize(im, (224, 224)).astype(np.float32)

#im /= 255.0
#im = im - np.mean(im, axis=2, keepdims=True)

im = np.expand_dims(im, axis=0)
im = preprocess_input(im)

out = vgg16_model.predict(im)

It seemed that im /= 255.0 give very bad prediction. I commented it out and it started making good prediction. I also added preprocess_input(...) but that doesn't seem to affect prediction for the few random trials I did.

The question is that according to this great blog:


under "Using the bottleneck features of a pre-trained network: 90% accuracy in a minute", pre-trained VGG16 is in a transfer learning context. And if you look at this gist, you see this line of code:

def save_bottlebeck_features():
    datagen = ImageDataGenerator(rescale=1. / 255)

The preprocessing of input seemed to be 1/255.0 during caching of features from the last conv layer. This is sort of puzzling. I further also looked up how preprocess_input(...) is defined in the code I have, and found for 'tf':

x /= 127.5
x -= 1.

Which you can check here.

  • $\begingroup$ I would like to add a word of caution. If you play around with this code, be aware that preprocess_input actually modify its input inplace since it is passed by reference. preprocess_input doesn't attempt to make a copy before manipulating the image. $\endgroup$ Apr 21 '18 at 18:28
  • $\begingroup$ Did you ever find a good anser as to why the input is scaled to -1 to 1 when using vgg16 on keras with tensorflow backend? $\endgroup$ Oct 17 '18 at 21:00
  • 1
    $\begingroup$ @user3731622, see my comment below. This bit of detail doesnt really matter, in the context of using pre-trained weights and transfer learning, as long as your preprocessing is sensible (i.e. bounded in O(1) and with mean O(1).) In my case, other aspect of training matter much more in getting good results. $\endgroup$ Nov 12 '18 at 4:40

The pre-trained weights that are available on Keras are trained with the preprocessing steps defined in preprocess_input() function that is made available for each network architecture (VGG16, InceptionV3, etc).

For example

from keras.applications.vgg16 import preprocess_input

If you are using the weights that comes with keras for fine tuning, then you should use the corresponding preprocess_input() function for the network. It can be different from the original preprocessing steps mentioned in the paper.

from keras.applications.vgg16 import VGG16
model = VGG16(weights='imagenet', include_top=False)

If you want to stick to the original preprocessing steps, you can find pre-trained weights that is trained with the original preprocessing steps instead of using the weights that comes with Keras.

Maybe at the time of writing the blog post the weights were trained with different preprocessing steps.

Checkout this github issue to learn more.

  • $\begingroup$ I don't think this is a big deal. In my experience now, you can use a preprocessing thats not too far from the original one. In my original concern, it comes down to values bounded between 0 and 1, or -1 and 1. In either case, the one that drives the test accuracy is your dataset, regularization, data augmentation, etc, etc. (for my work, i obtained a 98% test accuracy), and this minor detail just doesnt matter. $\endgroup$ Nov 12 '18 at 4:37

I have a new thinking on this. I think it maybe ok to use a different but reasonable preprocessing (such as 1/255.) in the context of transfer learning (pre-training), than whats originally used to train the VGG16. As long as the top most conv-conv-...pool representation is useful for your new task, and it empirically checked out fine for what F. Chollet did in his blog.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.