Different results between ImageDataGenerator and model.predict in Tensorflow

Question

I am training a simple cnn using flow from directory with train and validation datasets. The dataset pattern are as follows,

Train_dataset  
----good_data  
      ----good_image_01.png
      ----good_image_02.png
----bad_data  
    ----bad_image_01.png
    ----bad_image_02.png

Validation_dataset  
----good_data  
      ----good_image_01.png
      ----good_image_02.png
----bad_data  
    ----bad_image_01.png
    ----bad_image_02.png

I am using the following model structure,

model = tf.keras.models.Sequential([
    tf.keras.layers.Conv2D(64, (3, 3), activation='relu', input_shape=(300, 300, 3)),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Conv2D(64, (3, 3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2, 2),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation=tf.nn.relu),
    tf.keras.layers.Dense(2, activation=tf.nn.softmax)])

model.compile(optimizer=tf.optimizers.Adam(),
              loss='categorical_crossentropy',
              metrics=['accuracy'])

and the generator looks as follows,

TRAINING_DIR = "Train/"
train_datagen = ImageDataGenerator(rescale=1. / 255,
                                   rotation_range=40,
                                   width_shift_range=0.2,
                                   height_shift_range=0.2,
                                   shear_range=0.2,
                                   zoom_range=0.2,
                                   horizontal_flip=True,
                                   fill_mode='nearest')
train_generator = train_datagen.flow_from_directory(TRAINING_DIR,
                                                    batch_size=32,
                                                    class_mode='categorical',
                                                    target_size=(300, 300))

VALIDATION_DIR = "Validation/"
validation_datagen = ImageDataGenerator(rescale=1. / 255,
                                        rotation_range=40,
                                        width_shift_range=0.2,
                                        height_shift_range=0.2,
                                        shear_range=0.2,
                                        zoom_range=0.2,
                                        horizontal_flip=True,
                                        fill_mode='nearest')
validation_generator = validation_datagen.flow_from_directory(VALIDATION_DIR,
                                                              batch_size=32,
                                                              class_mode='categorical',
                                                              target_size=(300, 300))

I fit the generator as follows,

checkpoint_path = "models/cp.ckpt"
cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_path,
                                                 save_weights_only=True,
                                                 verbose=1)
history = model.fit(train_generator,
                    epochs=150,
                    verbose=1,
                    validation_data=validation_generator, callbacks=[cp_callback])

and the training struggle to optimize initially but after 100 epochs the network learns and the training reaches an accuracy of 93% on both train and validation split.

The issue is when I load the saved weights and predict the result for both the classes , i get all the predictions with argmax on the first class i.e every image from validation gets classified as good class. The code i use for prediction is,

model = create_model()
checkpoint_path = "models/cp.ckpt"

model.load_weights(checkpoint_path)

test_path = 'Validation/bad_data'

for item in os.listdir(test_path):
    full_path = os.path.join(test_path, item)
    img = cv2.imread(full_path)
    img = cv2.resize(img, (300, 300))
    #print(img.shape)
    img = np.array(img)
    img = img.astype('float')
    # normalize to the range 0-1
    img /= 255.0
    img = np.expand_dims(img, axis=0)
    pred = model.predict(img)
    print(pred)

The model structure is same as during training. Can anyone help me on this?

Answer 1

I have found the answer. It lies with the way image is preprocessed before it is feeded on the prediction pipeline. Originally I used,

for item in os.listdir(test_path):
    full_path = os.path.join(test_path, item)
    img = cv2.imread(full_path)
    img = cv2.resize(img, (300, 300))
    #print(img.shape)
    img = np.array(img)
    img = img.astype('float')
    # normalize to the range 0-1
    img /= 255.0
    img = np.expand_dims(img, axis=0)
    pred = model.predict(img)
    print(pred)

In which I used Opencv to preprocess the image. But I tried playing around with other libraries such as Pillow, skiimage imageio but nothing worked until i finally used tf.keras.preprocessing.image

for item in os.listdir(test_path):
    full_path = os.path.join(test_path, item)
    img = tf.keras.preprocessing.image.load_img(full_path, target_size=(300, 300))
    img_tensor = tf.keras.preprocessing.image.img_to_array(img)  # (height, width, channels)
    img_tensor = np.expand_dims(img_tensor, axis=0) 
    img_tensor /= 255. 
    pred = model.predict(img_tensor)
    print(pred)