# Issue with predict generator keras

I'm new with keras with tensorflow backend and I'm trying to do transfer learning with pretrained net. The problem is that the accuracy on validation set is very high, around the 90% , but on test set the accuracy is very bad, less that 1%. I solved the problem using opencv to read and resize image, but I'd like to understand why with keras methods I've this problem. I paste my code below.

from keras.preprocessing.image import ImageDataGenerator
from keras.applications.xception import preprocess_input
import keras

train_val_datagen = ImageDataGenerator( validation_split=0.25, preprocessing_function=preprocess_input)

train_val_generator = train_val_datagen.flow_from_directory( # subset di allenamento
directory="./image-dataset/",
target_size=(299, 299),
color_mode="rgb",
batch_size=32,
class_mode="categorical",
shuffle=True,
subset = 'training',
seed=17)
val_train_generator = train_val_datagen.flow_from_directory( # subset di validation
directory="./image-dataset/",
target_size=(299, 299),
color_mode="rgb",
batch_size=32,
class_mode="categorical",
shuffle=True,
subset = 'validation',
seed=17)
final_train_generator = train_val_datagen.flow_from_directory( # set finale di allenamento con tutti i dati
directory="./image-dataset/",
target_size=(299, 299),
color_mode="rgb",
batch_size=32,
class_mode="categorical",
shuffle=True,
seed=17)


As you can see i used Xception as pretrained-net and I choose to resize my images to adapt them to the net.

After the training I created a new Iterator for test data as follow:

test_datagen = ImageDataGenerator(preprocessing_function=preprocess_input)
test_generator = test_datagen.flow_from_directory(
directory="./TEST/",
target_size=(299, 299),
color_mode="rgb",
batch_size=1, # predico una alla volta
shuffle = False,
class_mode=None # non ce alcuna classe di riferimento
)
test_generator.reset()


where the preprocess function is exactly the same.

The predictions are made with the following code:

predictions = model_xcpetion.predict_generator(test_generator, 6104, verbose = 1 )


where 6104 is the number of images in test folder. After this i've generated a csv with images name with relative categorical probabilities:

import pandas as pd
import numpy as np
df = pd.DataFrame(predictions)
cols =[('probability of' + str(i)) for i in list(range(1, 30 )) ]
df.columns =  cols
df['images'] = imNames
df.to_csv('predictions_xception_all_data.csv', sep=',')


where the columns represent the labels (1 to 29) and imNames are obtained with filenames attribute of test_generator. Finally I generated the csv with the labels with highest probabilities value and i compute the accuracy obtaining the value that I wrote before.

The code that I used to solve is the same but I read and resize images with the following code:

width = 299
height = 299
dim = (width, height)
images = []
# for each img
resized = cv2.resize(img, dim, interpolation = cv2.INTER_AREA)
images.append(resized)



EDIT1: the images resized with opencv have not been processed with the preprocessing function

• I did a test generator like in the following link. Maybe it helps stackoverflow.com/questions/52270177/… Dec 23, 2019 at 23:09
• Just a suggestion. Keras has stopped developing and has been integrated with tensorflow 2.0. Since you're only starting, I would suggest you start to you tf2.0. It has Keras within it. So, you can just replace import keras with from tensorflow import keras and all of your code will work fine. Dec 26, 2019 at 3:42
• @NagabhushanSN thanks for suggestion. I'll move on tensorflow 2.0 Dec 26, 2019 at 11:35
• @AgostinoDorano: Cool! Maybe you can post your code/solution here as an answer to your question, so that people can learn from it? Cheers Dec 26, 2019 at 11:51
• @Peter I just posted it. regards! :) Dec 26, 2019 at 12:17

I solved the problem following the advices in the comments of this discussion. I paste here my code:

dizionario = dict({'1': 0,
'10': 1,
'11': 2,
'12': 3,
'13': 4,
'14': 5,
'15': 6,
'16': 7,
'17': 8,
'18': 9,
'19': 10,
'2': 11,
'20': 12,
'21': 13,
'22': 14,
'23': 15,
'24': 16,
'25': 17,
'26': 18,
'27': 19,
'28': 20,
'29': 21,
'3': 22,
'4': 23,
'5': 24,
'6': 25,
'7': 26,
'8': 27,
'9': 28})


as you can see I created a dictionary to map classes label to index . I used the output of final_train_generator.class_indices to create this dictionary. After predict_generator I created the csv of prediction with following code:

predicted_class_indices = np.argmax(predictions, axis = 1)
final_predictions = []
for element in predicted_class_indices:
final_predictions.append(list(dizionario.keys())[list(dizionario.values()).index(element)])
df = pd.DataFrame()
df['class'] = final_predictions
df['imnames'] = imNames
df.to_csv('predictions_xception_all_data_bon.csv', sep=',')


I paste here also the code that I changed following the discussion link in the comments:

test_datagen = ImageDataGenerator(preprocessing_function=preprocess_input)
test_generator = test_datagen.flow_from_directory(
directory="./TEST/",
target_size=(299, 299),
color_mode="rgb",
batch_size=20,
shuffle = False,
class_mode = "categorical",
)
test_generator.reset()
imNames = test_generator.filenames
predictions = model_xcpetion.predict_generator(test_generator, steps=len(test_generator), verbose = 1 )


This behavior is because flow from directory assigns class labels lexicographically. So, all the folder names are treated not as int but string.

This is why you see folder '1' has class label 0. Folder '10' has class label 1 rather than Folder '2'.