why validation accuracy is stuck at 75%?

Question

i am using tensorflow=2.15.0 and keras associated with it I have made a cnn network to identify a total of 2294 images into 10 different classes or, data is divided as 229 images are contained in each folder and the images are sorted or numbered according to the timestamp. size of image is 300(width),900(height),1(grayscale)

I have read almost all of the posts regarding this in stackoverflow and other online forums but I do not get the answer for my case

I have tried every possible thing except crossvalidation using k-fold because I cannot figure out how to apply it in my case as the former code before the creation of model is little bit different as what is taught in the example.

 # Train-Test Split
X_train, X_test, Y_train, Y_test = train_test_split(all_images, all_labels_one_hot, test_size=0.3, random_state=99,shuffle=True)
print('X_train.shape:', X_train.shape)
print('X_test.shape:', X_test.shape)
print('Y_train.shape:', Y_train.shape)
print('Y_test.shape:', Y_test.shape)

 # Output
X_train.shape: (1605, 900, 300)
X_test.shape: (689, 900, 300)
Y_train.shape: (1605, 10)
Y_test.shape: (689, 10)

# Define the model
model = keras.Sequential([
    keras.layers.Conv2D(filters=32, kernel_size=3, activation=keras.layers.LeakyReLU(alpha=0.01), kernel_initializer='he_uniform', padding = 'same', input_shape=img_shape, name='conv_01'),
    keras.layers.BatchNormalization(),
    keras.layers.AveragePooling2D(pool_size=2,strides=2, name='pool_01'),

    keras.layers.Conv2D(filters=64, kernel_size=3,activation=keras.layers.LeakyReLU(alpha=0.01),kernel_initializer='he_uniform', padding='same', name='conv_02'),
    keras.layers.BatchNormalization(), 
    keras.layers.AveragePooling2D(pool_size=2,strides=2, name='pool_02'),  
    
    keras.layers.Conv2D(filters=64, kernel_size=3,activation=keras.layers.LeakyReLU(alpha=0.01),kernel_initializer='he_uniform', padding='same', name='conv_03'),
    keras.layers.BatchNormalization(),   
    keras.layers.AveragePooling2D(pool_size=2, strides=2,name='pool_03'),

    keras.layers.Conv2D(filters=128, kernel_size=3,activation=keras.layers.LeakyReLU(alpha=0.01), kernel_initializer='he_uniform',padding='same', name='conv_04'),
    keras.layers.BatchNormalization(),   
    keras.layers.AveragePooling2D(pool_size=2, strides=2,name='pool_05'),  

 keras.layers.Flatten(name='flatten_01'),
    keras.layers.Dropout(0.3, name='dropout_01'),

    keras.layers.Dense(64,activation=keras.layers.LeakyReLU(alpha=0.01),kernel_regularizer=keras.regularizers.l2(l2=0.01),name='dense-03'),
    keras.layers.BatchNormalization(),
    keras.layers.Dropout(0.2, name='dropout_04'),

    keras.layers.Dense(128,activation=keras.layers.LeakyReLU(alpha=0.01),kernel_regularizer=keras.regularizers.l2(l2=0.01),name='dense-01'),
    keras.layers.BatchNormalization(),
    keras.layers.Dropout(0.2, name='dropout_02'),

    keras.layers.Dense(128,activation=keras.layers.LeakyReLU(alpha=0.01),kernel_regularizer=keras.regularizers.l2(l2=0.01),name='dense-02'),
    keras.layers.BatchNormalization(),
    keras.layers.Dropout(0.2, name='dropout_03'),

keras.layers.Dense(10, activation='softmax',name='fc_layer'),
])

#Compile the model
history = model.fit(X_train, Y_train, batch_size=16, epochs=50, validation_split=0.35,shuffle=True,callbacks=([lr_scheduler]))

optimizer used is RMSprop and loss is categorical cross entropy. Could anyone please suggest how can I avoid overfitting?

I have tried dataaugmentation, applied regularization, changing the hyperparameters but no effect. I have tried with the 2 and 3 conv2D layers with the change in filters but no improvement. Could someone please help?

in addition to the question, I have changed the architecture so initially 3 conv2D -> 32/64/128 followed by one hidden layer(128) and the last output layer of 10 neurons so I am getting validation accuracy of 89% and training accuracy of 99%

and again i have changed the architecture so 3 conv2D -> 32/64/128 followed by two hidden layers(64/64) then one output layer of 10 neurons and the validation accuracy obtained is 85% and training accuracy obtained is 99%

I have put the image for the above run.

Answer 1

My first inclination is that your dataset size is relatively small, and you have many hidden layers, which increases the likelihood of your model overgeneralizing on the training data, leading to overfitting. Your training graphs of the loss and accuracy for train/test data confirm this.

Thoughts:

• Reduce Hidden Layers in size and quantity and compare the results of each run.
• Try reducing the Conv2D layers and sizes if the above does not work.
• Experiment with further data augmentation. Random rotations, flips, zooming, etc.
• Ensemble; however, this is a bit more complex of an approach.
• Experiment with different L2/L1 regulization techniques.