# Validation Accuracy stays constant upto 4 decimal places while Training Accuracy increases

This is my model summary for reference:

 Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d (Conv2D)              (None, 98, 98, 32)        320
_________________________________________________________________
activation (Activation)      (None, 98, 98, 32)        0
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 49, 49, 32)        0
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 47, 47, 32)        9248
_________________________________________________________________
activation_1 (Activation)    (None, 47, 47, 32)        0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 23, 23, 32)        0
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 21, 21, 64)        18496
_________________________________________________________________
activation_2 (Activation)    (None, 21, 21, 64)        0
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 10, 10, 64)        0
_________________________________________________________________
flatten (Flatten)            (None, 6400)              0
_________________________________________________________________
dense (Dense)                (None, 64)                409664
_________________________________________________________________
activation_3 (Activation)    (None, 64)                0
_________________________________________________________________
dropout (Dropout)            (None, 64)                0
_________________________________________________________________
dense_1 (Dense)              (None, 26)                1690
_________________________________________________________________
activation_4 (Activation)    (None, 26)                0
=================================================================
Total params: 439,418
Trainable params: 439,418
Non-trainable params: 0
_________________________________________________________________


This is the output i am getting when I train my model:

WARNING:tensorflow:sample_weight modes were coerced from
...
to
['...']
WARNING:tensorflow:sample_weight modes were coerced from
...
to
['...']
Train for 41 steps, validate for 8 steps
Epoch 1/5
41/41 [==============================] - 29s 719ms/step - loss: 0.4022 - acc: 0.8336 - val_loss: 0.1885 - val_acc: 0.9615
Epoch 2/5
41/41 [==============================] - 20s 480ms/step - loss: 0.2523 - acc: 0.9294 - val_loss: 0.1723 - val_acc: 0.9615
Epoch 3/5
41/41 [==============================] - 20s 481ms/step - loss: 0.2204 - acc: 0.9485 - val_loss: 0.1672 - val_acc: 0.9615
Epoch 4/5
41/41 [==============================] - 20s 482ms/step - loss: 0.2098 - acc: 0.9548 - val_loss: 0.1647 - val_acc: 0.9615
Epoch 5/5
41/41 [==============================] - 20s 498ms/step - loss: 0.1997 - acc: 0.9583 - val_loss: 0.1669 - val_acc: 0.9615


This is the model history after I trained the model:

{'loss': [0.40232268926556125, 0.2524432904698366, 0.22039496203873055, 0.2098230598894365, 0.199863911314976],

'acc': [0.83358896, 0.9294478, 0.9485017, 0.9548135, 0.958294],

'val_loss': [0.18854963406920433, 0.17227689549326897, 0.16715877316892147, 0.16466446034610271, 0.16689967922866344],

'val_acc': [0.9615384, 0.9615384, 0.9615384, 0.9615384, 0.9615384]}


The val_acc stays at 0.9615384 no matter what i do. I have tried increasing the number of epochs, my dataset is unbiased and i am using data augmentation so i don't think the amount of data is the problem.

NOTE: This is my first time using ImageDataGenerator(). The error could be because i do not use the proper use of generators.

The code i used to compile my model:

model.compile(
loss='binary_crossentropy',
metrics=['acc']
)


The code i used to train my model:

model_hist = model.fit( train_generator,
steps_per_epoch=train.shape[0] // batch_size,
epochs=5,
validation_data=validation_generator,
validation_steps=validate.shape[0] // batch_size)


I have used the flow_from_dataframe() function to create the generator form a dataframe called 'train'

Can someone please help me on how to resolve the problem? Or let me know where am I going wrong?

What is the number of samples? If the dataset has relatively small number of samples and some of them are much harder to classify then the others, it might cause the model to converge on a optimum that misclassifies them. I would recommend checking which samples are misclassified when you achieve this accuracy for each epoch and confirming if they are the same.

The loss goes down when the model converges to the optimum and classifies the "easy" samples correctly with more and more certainty.

Hope that helps,

Max

• The sample size is 668 for train set and 144 for validate set. I have tried changing the seed to see if the problem is due to improper data distribution, but I am gettin this 96% accuracy for all seeds. – Harsh Raj Jul 21 at 13:02
• First check if these are the same samples which are misclasified. Then, check which samples are consistently misclasified and try to see if there is something different about them then the other samples from the dataset. I see you have 26 classes so probably you are classifying letters? It might be that one of the classes is always misclassified because you might have not included it in the training dataset. – maksym33 Jul 21 at 13:23
• First of all. Thankyou so much for your help. I found out my classifier is always predicting a "W" and it still shows 96% accuracy. I have no clue why though.. – Harsh Raj Jul 21 at 13:41
• Now I see, try to use categorical accuracy instead. See keras.io/api/metrics/accuracy_metrics . Regarding the learning, you are using binary CE while it should be categorical CE (see keras.io/api/losses/probabilistic_losses) unless you are using multilabel dataset (two letters on one picture). – maksym33 Jul 21 at 13:44
• So, I should change my metrics and loss both? Can you please guide me to a resource that can help me with misunderstandings like this. Also, where and how did you learn? – Harsh Raj Jul 21 at 13:51