1
$\begingroup$

I have a problem. I have trained a model. And as you can see, there is a zigzag in the loss. In addition, the validation loss is increasing. What does this mean if you only look at the training curve? Is there an overfitting?

And the model does not generalise. Accuarcy on test and val is 0.84 and on the test 0.1. Does the assumption confirm the overfitting? And can overfitting come from the fact that I have trained too little? I only used two dense layers.

enter image description here

$\endgroup$
3
  • 1
    $\begingroup$ One or more of your hyperparameter values may be off. Too large learning rate? $\endgroup$
    – noe
    Oct 27, 2022 at 9:35
  • 1
    $\begingroup$ Overfitting is when there is too much training and/or not enough training data. As I said on your other similar question, it's not normal that the val accuracy is higher than the training. Please give more detail about your data: size, number of classes and proportion by class. Imho this looks like you're counting that a complex system will compensate bad/small data... if so it won't work, I'm afraid. $\endgroup$
    – Erwan
    Oct 27, 2022 at 10:59
  • $\begingroup$ Does this answer your question? Loss is very erratic in the 100s and val_loss is at 0, something - what is the reason for that? $\endgroup$ Oct 27, 2022 at 13:46

1 Answer 1

3
$\begingroup$

Please notice that your loss oscillates between 175 and zero. In which case I would look for potential problems in the code with respect to

  • loss calculation
  • batch size (increase)
  • train/validation set split strategy (stratification wrt class)

In a more general sense:

  • size of your network may be small
  • activation function saturation (avoid saddle points - use relu)
  • learning rate
  • normalisation before training

I hope these are helpful as a starting point. I would like to also point to this resource wrt training and fine tuning a deep learning model.

Hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.