# Pre-trained CNN model makes Poor Predictions on Test Images Dataset

I have tried using several a pretrained models (MobileNet) for multiclass predictions. There are 42 classes and the distributions of the images are even across the 42 classes.

This is my code:

base_model=MobileNet(weights='imagenet',include_top=False,input_shape = (224,224,3)) #imports the mobilenet model and discards the last 1000 neuron layer.
x=base_model.output
x=GlobalAveragePooling2D()(x)
x=Dense(512,activation='relu')(x) #we add dense layers so that the model can learn more complex functions and classify for better results.
preds=Dense(42,activation='softmax')(x) #final layer with softmax activation
model=Model(inputs=base_model.input,outputs=preds)
for layer in base_model.layers[:20]:
layer.trainable=False
for layer in base_model.layers[20:]:
layer.trainable=True


I have freezed and unfreezed some of the trainable layers.

I let it run for 5 epochs:

Epoch 1/5
1318/1318 [==============================] - 3604s 3s/step - loss: 1.5493 - accuracy: 0.5796 - val_loss: 1.7180 - val_accuracy: 0.5361
Epoch 2/5
1318/1318 [==============================] - 3272s 2s/step - loss: 1.2174 - accuracy: 0.6641 - val_loss: 1.7562 - val_accuracy: 0.5372
Epoch 3/5
1318/1318 [==============================] - 3233s 2s/step - loss: 1.0853 - accuracy: 0.6981 - val_loss: 1.2993 - val_accuracy: 0.6498
Epoch 4/5
1318/1318 [==============================] - 3223s 2s/step - loss: 0.9918 - accuracy: 0.7224 - val_loss: 1.3455 - val_accuracy: 0.6382
Epoch 5/5
1318/1318 [==============================] - 3310s 3s/step - loss: 0.9153 - accuracy: 0.7413 - val_loss: 1.2375 - val_accuracy: 0.6660


The accuracy I got is relatively good. I have tried with other pretrained models like Xception too and the accuracies were pretty good. However when I use my model to predict on the test data, and upload my submission to Kaggle, my predictions are very bad.

I am wondering if I have imported the test images and made predictions correctly? Here is the code:

import cv2
pred_images = []
filename= []

for image_file in os.listdir('./test/test'): #Extracting the file name of the image from Class Label folder
filename.append(image_file)
image = cv2.resize(image,(224,224)) #Resize the image, Some images are different sizes. (Resizing is very Important)
pred_images.append(image)

category = []
for i in range(len(pred_images)):
pred_image = np.array([pred_images[i]])
pred = model.predict(pred_image)
cat = np.argmax(pred, axis = 1)
category.extend(cat)


I get a data frame looking like this, with a lot from class 38 and 20. Is there something wrong?

final = pd.DataFrame(list(zip(filename, category)), columns = ['filename', 'category'])

    filename                               category
0   c94de2fa9b06d67848f648e33a43475c.jpg    38
1   bbb7a2da148488bb878727556aa5914c.jpg    38
2   606256bdf3636d280bfdc3def33a57e7.jpg    38
3   7f741619b952876e7c7c419a0de1ed60.jpg    20
4   05c7592b31ceb8e14d4faa30fa21794c.jpg    20


Kaggle's Dataset is really noisy and it is really tough to get very good result with simple models. You have to do lots of preproccesing ,Data Augmentations and larger network for many more epoch to get a good result. Things you can do :

1. Go to notebook section of your competetion and look out for some preprocessing and augmentations and apply them.
2. Use some regularizer like dropout, mixup,cutmix and many more.

3.Train your model for more epoch

1. Train a larger model.
• I have done augmentations with the help of ImageDataGenerator already. Even though I train for many epochs, the accuracy on the test set is still super bad, even though the val_accuracy is around 0.7. The model I believe is large enough around 100,000 images. – eun ji Jul 1 '20 at 9:06
• So there is nothing wrong with the way I use my model to predict on the test images? – eun ji Jul 1 '20 at 9:07
• image = cv2.resize(image,(224,224,3)) use 3 as your channel as input dimension is (224,224,3) not (224,224) and do necessary change to keep the dim (224,224,3) while inputting in model – SrJ Jul 1 '20 at 9:09
• That did not affect anything. – eun ji Jul 1 '20 at 9:13
• Kaggle dataset is really hard to get good level of score. You should increase your model's depth and train it for at least 30 epoch with proper regulazier – SrJ Jul 1 '20 at 9:15