1
$\begingroup$

So I have a model to classify images into 3 classes. I have 10.5k train images ( 3.5 per each category ) and 3k ( 1k per each category ) validation images but I can't increase my val_acc no matter what I try. I will put the whole code down below, in case I do something wrong processing the data.

I started off with a simple base model:

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(224, 224, 3)),
    tf.keras.layers.Rescaling(1./255),
    tf.keras.layers.Conv2D(16, 3, padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Conv2D(32, 3, padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Conv2D(64, 3, padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(3)
    ])
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

this, without any data preprocessing like rotations, scaling and so on, and no dropouts, resulted in close to 0.9 train_acc, but only around 0.3 val_acc so I clearly saw this as an overfitting sign.

for 15 epochs:

epochs = 15
history = model.fit(
  train_dataset,
  validation_data=val_dataset,
  epochs=epochs
)

this results in around 90 accuracy for train data, but the val_acc never grows. It stagnates around 0.34

165/165 ━━━━━━━━━━━━━━━━━━━━ 30s 98ms/step - accuracy: 0.3933 - loss: 1.4266 - val_accuracy: 0.3377 - val_loss: 1.1598
Epoch 2/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.5214 - loss: 0.9857 - val_accuracy: 0.3497 - val_loss: 1.2092
Epoch 3/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.5849 - loss: 0.9057 - val_accuracy: 0.3513 - val_loss: 1.3415
Epoch 4/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.6281 - loss: 0.8238 - val_accuracy: 0.3470 - val_loss: 1.5618
Epoch 5/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 56ms/step - accuracy: 0.6877 - loss: 0.7123 - val_accuracy: 0.3460 - val_loss: 1.8965
Epoch 6/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.7389 - loss: 0.6132 - val_accuracy: 0.3390 - val_loss: 2.1827
Epoch 7/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.7600 - loss: 0.5644 - val_accuracy: 0.3467 - val_loss: 2.2611
Epoch 8/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.8277 - loss: 0.4246 - val_accuracy: 0.3347 - val_loss: 2.5835
Epoch 9/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.8561 - loss: 0.3664 - val_accuracy: 0.3447 - val_loss: 2.6086
Epoch 10/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 56ms/step - accuracy: 0.8892 - loss: 0.2964 - val_accuracy: 0.3380 - val_loss: 3.0506
Epoch 11/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.9051 - loss: 0.2513 - val_accuracy: 0.3350 - val_loss: 3.1805
Epoch 12/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.9211 - loss: 0.2048 - val_accuracy: 0.3413 - val_loss: 3.9698
Epoch 13/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.9407 - loss: 0.1620 - val_accuracy: 0.3350 - val_loss: 4.6286
Epoch 14/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 55ms/step - accuracy: 0.9620 - loss: 0.1211 - val_accuracy: 0.3407 - val_loss: 5.1370
Epoch 15/15
165/165 ━━━━━━━━━━━━━━━━━━━━ 9s 56ms/step - accuracy: 0.9679 - loss: 0.0912 - val_accuracy: 0.3460 - val_loss: 5.4826

Changes I made:

  1. Different batch sizes, different image sizes, different preprocessing augmentations
  2. Adding dropout
  3. Adding regularizations, like l2 norm
  4. Resizing layers sizes

None of this actually works, and the val_accuracy gets stuck somewhere around 0.32-0.35 so I can't stop thinking about what am I doing wrong here.

latest changes to the model:

data_augmentation = tf.keras.Sequential(
  [
    
    tf.keras.layers.RandomFlip("horizontal"),
    tf.keras.layers.RandomRotation(0.1),
    tf.keras.layers.RandomZoom(0.1),
  ]
)

def create_model():
  model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(80, 80, 3)),
    data_augmentation,
    tf.keras.layers.Rescaling(1./255),
    tf.keras.layers.Conv2D(64, 3, padding='same', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(0.0001)),
    #tf.keras.layers.Dropout(0.2),
    tf.keras.layers.MaxPooling2D(),
    #tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Conv2D(128, 3, padding='same', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(0.0001)),
    #tf.keras.layers.Dropout(0.2),
    tf.keras.layers.MaxPooling2D(),
    #tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Conv2D(512, 3, padding='same', activation='relu', kernel_regularizer=tf.keras.regularizers.l2(0.0001)),
    #tf.keras.layers.Dropout(0.2),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(512, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(3)
    ])
  optimizer = tf.keras.optimizers.Adam(learning_rate=0.0001)
  model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
       
              metrics=['accuracy'])

  return model

model = create_model()

With these changes, train_accuracy lowers to 0.7, but the val_accuracy is still at around 0.32 - 0.35.

Reading data:

import os
import pandas as pd
import tensorflow as tf
from tensorflow.keras.preprocessing.image import img_to_array, load_img
import numpy as np

# Set the paths to the competition's input directory
input_dir = '/image-classification'  

# Paths to CSV files and image directories
train_csv_file = os.path.join(input_dir, 'train.csv')  
val_csv_file = os.path.join(input_dir, 'validation.csv')  

train_image_dir = os.path.join(input_dir, 'train')  
val_image_dir = os.path.join(input_dir, 'validation') 
# Load the CSV files
train_df = pd.read_csv(train_csv_file)
val_df = pd.read_csv(val_csv_file)


# Function to load and preprocess images
def load_and_preprocess_image(image_id, img_dir, target_size=(224, 224)):
    img_path = os.path.join(img_dir, f"{image_id}.png")  
    img = load_img(img_path, target_size=target_size)  # Load the image with target size
    img_array = img_to_array(img)  # Convert the image to an array
   
    return img_array

# Function to load images and labels from a dataframe
def load_images_and_labels(df, img_dir):
    images = []
    labels = []
    for index, row in df.iterrows():
        image_id = row['image_id']
        label = row['label']
        try:
            image = load_and_preprocess_image(image_id, img_dir)
            images.append(image)
            labels.append(label)
        except FileNotFoundError:
            print(f"Image {image_id} not found, skipping.")
    return np.array(images), np.array(labels)


X_train, y_train = load_images_and_labels(train_df, train_image_dir)

# Load validation images and labels
X_val, y_val = load_images_and_labels(val_df, val_image_dir)

# Create TensorFlow datasets
train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))
val_dataset = tf.data.Dataset.from_tensor_slices((X_val, y_val))


# Batch and prefetch the datasets
train_dataset = train_dataset.batch(64).prefetch(buffer_size=tf.data.AUTOTUNE)
val_dataset = val_dataset.batch(64).prefetch(buffer_size=tf.data.AUTOTUNE)

EDIT, including adding multiple conv layers and changing the kernel

So as suggested, I added multiple conv layers and also tried changing the kernel size:

for example:

model = tf.keras.Sequential([
    #data_augmentation,
    tf.keras.layers.Rescaling(1./255, input_shape=(224,
                                  224,
                                  3)),
    tf.keras.layers.Conv2D(16, (4, 4), padding='same', activation='relu'),
    tf.keras.layers.Conv2D(16, (4, 4), padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Conv2D(32, (4, 4), padding='same', activation='relu'),
    tf.keras.layers.Conv2D(32, (4, 4), padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Conv2D(128, 3, padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(3)
    ])
  model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

  return model

this without data_augumentation, results into almost 100% train_acc, but again, the val_acc is stuck at 0.32-0.35

with data_augumentation applied, it gets to 70% train_acc and, of, course, 0.32-0.35 val_acc

$\endgroup$
6
  • $\begingroup$ It is difficult to find the specific reason without knowing what the data looks like, but have you tried adding more convolutional layers before going to a pooling layer? $\endgroup$
    – Oxbowerce
    Commented May 26 at 13:57
  • $\begingroup$ @Oxbowerce yes, I tried. I even tried different kernel sizes for the conv layer, i will edit the post to include that. What bothers me the most is that no matter what I try, the val_acc is somewhere near 0.33, almost like it would randomly predict for every photo instead of learning. This happens every time, it doesn't go behind0.32, not even at the start, and it doesn't go beyond 0.35 neither. $\endgroup$
    – Dragos123
    Commented May 26 at 16:15
  • $\begingroup$ What kind of images do you use? Would an untrained human easily get better accuracy? Also, have you tried varying the learning rate? $\endgroup$ Commented May 27 at 7:32
  • 1
    $\begingroup$ @picky_porpoise Images are 80x80 color, but as you guessed, an untrained human won't get a better accuracy. Matter of fact, I was not able to find any correlation between the images and class, so I wasn't either able to differentiate them. I tried varying the learning rate but it doesn't help. Also, something to keep an eye out for: answer of this question, about the activation. It didn't help for my val_accuracy, it's still stuck at 0.34 but at least now I can see the activation layers in my model. $\endgroup$
    – Dragos123
    Commented May 27 at 8:33
  • $\begingroup$ @Dragos123 In that case you might be out of luck. If you cannot differentiate the categories yourself, then it is unlikely a model like the one you show (i.e. not very deep) will easily do it. $\endgroup$ Commented May 27 at 14:13

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.