Why is val accuracy 100% within 2 epochs and incorrectly predicting new images? (1,000 images per class when training)

Question

My CNN tensorflow model reports 100% validation accuracy within 2 epochs. But it incorrectly predicts on single new images. (It is multiclass problem. I have 3 classes). How to resolve this? Can you please help me understand these epoch results?

I have 1,000 images per class that are representative of my testing data. How can validation accuracy reach 1.00 in just the first epoch when I have a dataset of 3,000 images in total, equal amount per class? (I would expect this to start at around 33% percent -- 1/ 3 classes.

Attempts:

1. I tried to ensure that I have correctly split my dataset into training and validation.
2. I understand overfitting can be a problem. I've added a dropout layer to try to solve this potential problem. From this questionhttps://ai.stackexchange.com/questions/5318/what-to-do-if-cnn-cannot-overfit-a-training-set-on-adding-dropout/23425 I learned that a "model is over-fitting if during training your training loss continues to decrease but (in the later epochs) your validation loss begins to increase. That means the model can not generalize well to images it has not previously encountered." I don't believe my model is overfitting based on this description. (My model reports both high training and high validation accuracy. If my model was overfitting I'd expect high training accuracy and low validation accuracy.)

My model:

def model():
  model_input = tf.keras.layers.Input(shape=(h, w, 3)) 
  x = tf.keras.layers.Rescaling(rescale_factor)(model_input) 
  x = tf.keras.layers.Conv2D(16, 3, activation='relu',padding='same')(x)
  x = tf.keras.layers.Dropout(.5)(x)
  x = tf.keras.layers.MaxPooling2D()(x) 
  x = tf.keras.layers.Flatten()(x)
  x = tf.keras.layers.Dense(128, activation='relu')(x)
  outputs = tf.keras.layers.Dense(num_classes, activation = 'softmax')(x)

Epoch results:

Epoch 1/10
/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:1096: UserWarning: "`sparse_categorical_crossentropy` received `from_logits=True`, but the `output` argument was produced by a sigmoid or softmax activation and thus does not represent logits. Was this intended?"
  return dispatch_target(*args, **kwargs)
27/27 [==============================] - 13s 124ms/step - loss: 1.0004 - accuracy: 0.5953 - val_loss: 0.5053 - val_accuracy: 0.8920
Epoch 2/10
27/27 [==============================] - 1s 46ms/step - loss: 0.1368 - accuracy: 0.9825 - val_loss: 0.0126 - val_accuracy: 1.0000
Epoch 3/10
27/27 [==============================] - 1s 42ms/step - loss: 0.0020 - accuracy: 1.0000 - val_loss: 5.9116e-04 - val_accuracy: 1.0000
Epoch 4/10
27/27 [==============================] - 1s 42ms/step - loss: 3.0633e-04 - accuracy: 1.0000 - val_loss: 3.5376e-04 - val_accuracy: 1.0000
Epoch 5/10
27/27 [==============================] - 1s 42ms/step - loss: 1.7445e-04 - accuracy: 1.0000 - val_loss: 2.2319e-04 - val_accuracy: 1.0000
Epoch 6/10
27/27 [==============================] - 1s 42ms/step - loss: 1.2910e-04 - accuracy: 1.0000 - val_loss: 1.8078e-04 - val_accuracy: 1.0000
Epoch 7/10
27/27 [==============================] - 1s 42ms/step - loss: 1.0425e-04 - accuracy: 1.0000 - val_loss: 1.4247e-04 - val_accuracy: 1.0000
Epoch 8/10
27/27 [==============================] - 1s 42ms/step - loss: 8.6284e-05 - accuracy: 1.0000 - val_loss: 1.2057e-04 - val_accuracy: 1.0000
Epoch 9/10
27/27 [==============================] - 1s 42ms/step - loss: 7.0085e-05 - accuracy: 1.0000 - val_loss: 9.3485e-05 - val_accuracy: 1.0000
Epoch 10/10
27/27 [==============================] - 1s 42ms/step - loss: 5.4979e-05 - accuracy: 1.0000 - val_loss: 8.5952e-05 - val_accuracy: 1.0000

Model.fit and model.compile:

model = model()

model = tf.keras.Model(model_input, outputs)
  
 model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['accuracy'])
  
hist = model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=10
)

Code to predict new image:

def makePrediction(image):
  from IPython.display import display
  from PIL import Image
  from tensorflow.keras.preprocessing import image_dataset_from_directory 
  img = keras.preprocessing.image.load_img(
  image, target_size=(h, q)
  )
  img_array = keras.preprocessing.image.img_to_array(img)
  img_array = tf.expand_dims(img_array, 0) #Create a batch
 
  predicts = model.predict(img_array)
  p = class_names[np.argmax(predicts)]
  return p

Going to the "data" directory and using the folders to create a dataset. Each folder is a class label:

from keras.preprocessing import image
directory_data = "data"
tf.keras.utils.image_dataset_from_directory(
    directory_testData, labels='inferred', label_mode='int',
    class_names=None, color_mode='rgb', batch_size=32, image_size=(256,
    256), shuffle=True, seed=123, validation_split=0.2, subset="validation",
    interpolation='bilinear', follow_links=False,
    crop_to_aspect_ratio=False
)
 
tf.keras.utils.image_dataset_from_directory(directory_testData, labels='inferred')

Creating dataset and splitting it:

Train_ds code: (Output: Found 1605 files belonging to 3 classes. Using 1284 files for training.)

train_ds = tf.keras.preprocessing.image_dataset_from_directory(
  directory_data = "data",
  validation_split=0.2,
  subset="training",
  seed=123,
  image_size=(h, w),
  batch_size=batch_size)

Val_ds code: (Output: Found 1605 files belonging to 3 classes. Using 321 files for validation.)

val_ds = tf.keras.preprocessing.image_dataset_from_directory(
directory_data = "data",
  validation_split=0.2,
  subset="validation",
  seed=123,
  image_size=(h, w),
  batch_size=batch_size)

Answer 1

In model.compile you are using the wrong metrics (metrics=['accuracy'])... As you are using SparseCategoricalCrossentropy you should use SparseCategoricalAccuracy.

from tensorflow.keras.metrics import SparseCategoricalAccuracy 

model.compile(loss=...,
              optimizer=..., 
              metrics=[SparseCategoricalAccuracy()])