1
$\begingroup$

I have a image classification model with 8400 images of class A and 1800 images of class B. I have used validation_split=0.2 with subsets of training and validation and batch sizes of 64.

I'm using a Sequential model with Augmentations and Conv2D, MaxPooling2D, BatchNormalization, Flatten, Dense, Dropout.

Since my dataset is not equal I'm calculating the following weights:

Weight for class A: 0.5629669156883672

Weight for class B: 4.470338983050848

I'm getting the following outputs:

enter image description here

Or in a different try I got these:

105/105 [==============================] - 185s 2s/step - loss: 2.5790 - accuracy: 0.8461 - val_loss: 0.3223 - val_accuracy: 0.8874
Epoch 2/30
105/105 [==============================] - 179s 2s/step - loss: 0.1982 - accuracy: 0.9368 - val_loss: 0.1439 - val_accuracy: 0.9392
Epoch 3/30
105/105 [==============================] - 179s 2s/step - loss: 0.1279 - accuracy: 0.9594 - val_loss: 0.0714 - val_accuracy: 0.9744
Epoch 4/30
105/105 [==============================] - 177s 2s/step - loss: 0.0651 - accuracy: 0.9769 - val_loss: 0.0127 - val_accuracy: 0.9952
Epoch 5/30
105/105 [==============================] - 177s 2s/step - loss: 0.0864 - accuracy: 0.9749 - val_loss: 0.0532 - val_accuracy: 0.9869
Epoch 6/30
105/105 [==============================] - 176s 2s/step - loss: 0.0622 - accuracy: 0.9824 - val_loss: 0.0483 - val_accuracy: 0.9863
Epoch 7/30
105/105 [==============================] - 176s 2s/step - loss: 0.0325 - accuracy: 0.9878 - val_loss: 0.0177 - val_accuracy: 0.9929
Epoch 8/30
105/105 [==============================] - 180s 2s/step - loss: 0.0255 - accuracy: 0.9942 - val_loss: 0.0229 - val_accuracy: 0.9917
Epoch 9/30
105/105 [==============================] - 184s 2s/step - loss: 0.0706 - accuracy: 0.9815 - val_loss: 0.0239 - val_accuracy: 0.9905
Epoch 9: early stopping

Can anyone help with these questions?

  • Is it normal that the first validation accuracy in multiple runs starts from maybe 80% in most cases? Sometimes higher than training accuracy.

  • Validation loss is decreasing in each epoch, but the validation accuracy just fluctuates, so I can't say it's improving. I have tried using kernel regularizers for this but it caused very high loss values.

  • What would be an acceptable loss and validation accuracy for such a question and model? Are these signs of some issue or just my dataset is too easy?

$\endgroup$

1 Answer 1

3
$\begingroup$

It's normal for the validation accuracy to start lower than the training accuracy, especially in the first few epochs. This is because the model is still learning and adjusting its weights to fit the training data better. As the training continues, the validation accuracy should improve and eventually converge to a stable value.

Fluctuations in validation accuracy are also common, especially if the dataset is small or the model is complex. It's important to look at the trend over multiple epochs rather than focusing on individual values. If the trend is generally upwards, then the model is likely learning and improving.

Regarding acceptable loss and validation accuracy, it depends on the specific problem and the desired level of performance. Generally, lower loss and higher accuracy are better, but it's important to consider the trade-off between accuracy and other factors such as computational resources and model complexity.

In terms of signs of issues, the fact that the validation loss is decreasing while the accuracy is fluctuating could indicate overfitting, where the model is fitting too closely to the training data and not generalizing well to new data. You could try increasing the regularization (such as dropout layer, early stopping or weight decay) or reducing the complexity of the model to address this. However, it's also possible that your dataset is relatively easy to classify, which could explain the high accuracy values.

$\endgroup$
2
  • $\begingroup$ Hey thanks for the answer, in the first paragraph I meant my initial validation accuracy is higher than training accuracy, which I think is strange, but gets better in next epochs. $\endgroup$
    – Amin Alaee
    Commented Feb 28, 2023 at 9:31
  • 1
    $\begingroup$ It's not uncommon for the validation accuracy to be higher than the training accuracy in the first epochs. So, if your training accuracy is improving over time and your validation accuracy is not decreasing, then it's likely that your model is learning and improving. It's also important to note that the difference in accuracy between the training and validation sets should not be too large. If the training accuracy is much higher than the validation accuracy, it could be a sign of overfitting. $\endgroup$
    – irazza
    Commented Feb 28, 2023 at 9:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.