1
$\begingroup$

My CNN tensorflow model reports 100% validation accuracy within 2 epochs. But it incorrectly predicts on single new images. (It is multiclass problem. I have 3 classes). How to resolve this? Can you please help me understand these epoch results?

I have 1,000 images per class that are representative of my testing data. How can validation accuracy reach 1.00 in just the first epoch when I have a dataset of 3,000 images in total, equal amount per class? (I would expect this to start at around 33% percent -- 1/ 3 classes.

Attempts:

  1. I tried to ensure that I have correctly split my dataset into training and validation.
  2. I understand overfitting can be a problem. I've added a dropout layer to try to solve this potential problem. From this questionhttps://ai.stackexchange.com/questions/5318/what-to-do-if-cnn-cannot-overfit-a-training-set-on-adding-dropout/23425 I learned that a "model is over-fitting if during training your training loss continues to decrease but (in the later epochs) your validation loss begins to increase. That means the model can not generalize well to images it has not previously encountered." I don't believe my model is overfitting based on this description. (My model reports both high training and high validation accuracy. If my model was overfitting I'd expect high training accuracy and low validation accuracy.)

My model:

def model():
  model_input = tf.keras.layers.Input(shape=(h, w, 3)) 
  x = tf.keras.layers.Rescaling(rescale_factor)(model_input) 
  x = tf.keras.layers.Conv2D(16, 3, activation='relu',padding='same')(x)
  x = tf.keras.layers.Dropout(.5)(x)
  x = tf.keras.layers.MaxPooling2D()(x) 
  x = tf.keras.layers.Flatten()(x)
  x = tf.keras.layers.Dense(128, activation='relu')(x)
  outputs = tf.keras.layers.Dense(num_classes, activation = 'softmax')(x)

Epoch results:

Epoch 1/10
/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:1096: UserWarning: "`sparse_categorical_crossentropy` received `from_logits=True`, but the `output` argument was produced by a sigmoid or softmax activation and thus does not represent logits. Was this intended?"
  return dispatch_target(*args, **kwargs)
27/27 [==============================] - 13s 124ms/step - loss: 1.0004 - accuracy: 0.5953 - val_loss: 0.5053 - val_accuracy: 0.8920
Epoch 2/10
27/27 [==============================] - 1s 46ms/step - loss: 0.1368 - accuracy: 0.9825 - val_loss: 0.0126 - val_accuracy: 1.0000
Epoch 3/10
27/27 [==============================] - 1s 42ms/step - loss: 0.0020 - accuracy: 1.0000 - val_loss: 5.9116e-04 - val_accuracy: 1.0000
Epoch 4/10
27/27 [==============================] - 1s 42ms/step - loss: 3.0633e-04 - accuracy: 1.0000 - val_loss: 3.5376e-04 - val_accuracy: 1.0000
Epoch 5/10
27/27 [==============================] - 1s 42ms/step - loss: 1.7445e-04 - accuracy: 1.0000 - val_loss: 2.2319e-04 - val_accuracy: 1.0000
Epoch 6/10
27/27 [==============================] - 1s 42ms/step - loss: 1.2910e-04 - accuracy: 1.0000 - val_loss: 1.8078e-04 - val_accuracy: 1.0000
Epoch 7/10
27/27 [==============================] - 1s 42ms/step - loss: 1.0425e-04 - accuracy: 1.0000 - val_loss: 1.4247e-04 - val_accuracy: 1.0000
Epoch 8/10
27/27 [==============================] - 1s 42ms/step - loss: 8.6284e-05 - accuracy: 1.0000 - val_loss: 1.2057e-04 - val_accuracy: 1.0000
Epoch 9/10
27/27 [==============================] - 1s 42ms/step - loss: 7.0085e-05 - accuracy: 1.0000 - val_loss: 9.3485e-05 - val_accuracy: 1.0000
Epoch 10/10
27/27 [==============================] - 1s 42ms/step - loss: 5.4979e-05 - accuracy: 1.0000 - val_loss: 8.5952e-05 - val_accuracy: 1.0000

Model.fit and model.compile:

model = model()

model = tf.keras.Model(model_input, outputs)
  
 model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['accuracy'])
  
hist = model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=10
)

Code to predict new image:

def makePrediction(image):
  from IPython.display import display
  from PIL import Image
  from tensorflow.keras.preprocessing import image_dataset_from_directory 
  img = keras.preprocessing.image.load_img(
  image, target_size=(h, q)
  )
  img_array = keras.preprocessing.image.img_to_array(img)
  img_array = tf.expand_dims(img_array, 0) #Create a batch
 
  predicts = model.predict(img_array)
  p = class_names[np.argmax(predicts)]
  return p

Going to the "data" directory and using the folders to create a dataset. Each folder is a class label:

from keras.preprocessing import image
directory_data = "data"
tf.keras.utils.image_dataset_from_directory(
    directory_testData, labels='inferred', label_mode='int',
    class_names=None, color_mode='rgb', batch_size=32, image_size=(256,
    256), shuffle=True, seed=123, validation_split=0.2, subset="validation",
    interpolation='bilinear', follow_links=False,
    crop_to_aspect_ratio=False
)
 
tf.keras.utils.image_dataset_from_directory(directory_testData, labels='inferred')

Creating dataset and splitting it:

Train_ds code: (Output: Found 1605 files belonging to 3 classes. Using 1284 files for training.)

train_ds = tf.keras.preprocessing.image_dataset_from_directory(
  directory_data = "data",
  validation_split=0.2,
  subset="training",
  seed=123,
  image_size=(h, w),
  batch_size=batch_size)

Val_ds code: (Output: Found 1605 files belonging to 3 classes. Using 321 files for validation.)

val_ds = tf.keras.preprocessing.image_dataset_from_directory(
directory_data = "data",
  validation_split=0.2,
  subset="validation",
  seed=123,
  image_size=(h, w),
  batch_size=batch_size)
$\endgroup$
2
  • 1
    $\begingroup$ Are you sure your image preprocessing for prediction follows the same steps as during training. Maybe have a look here (oldish Keras code) stackoverflow.com/questions/52270177/… $\endgroup$
    – Peter
    Nov 29, 2021 at 23:00
  • $\begingroup$ @Peter I believe I am following the same image preprocessing for both prediction and training. I've shown the preprocessing above in the makePrediction function and train_ds code. Do you see any inconsistencies here in the image processing? Thank you for the link :) $\endgroup$
    – user12342
    Nov 29, 2021 at 23:16

1 Answer 1

1
$\begingroup$

In model.compile you are using the wrong metrics (metrics=['accuracy'])... As you are using SparseCategoricalCrossentropy you should use SparseCategoricalAccuracy.

from tensorflow.keras.metrics import SparseCategoricalAccuracy 

model.compile(loss=...,
              optimizer=..., 
              metrics=[SparseCategoricalAccuracy()])
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.