What does that mean if the loss looks like this?

Question

I have a problem. I have trained a model. And as you can see, there is a zigzag in the loss. In addition, the validation loss is increasing. What does this mean if you only look at the training curve? Is there an overfitting?

And the model does not generalise. Accuarcy on test and val is 0.84 and on the test 0.1. Does the assumption confirm the overfitting? And can overfitting come from the fact that I have trained too little? I only used two dense layers.

Answer 1

Please notice that your loss oscillates between 175 and zero. In which case I would look for potential problems in the code with respect to

• loss calculation
• batch size (increase)
• train/validation set split strategy (stratification wrt class)

In a more general sense:

• size of your network may be small
• activation function saturation (avoid saddle points - use relu)
• learning rate
• normalisation before training

I hope these are helpful as a starting point. I would like to also point to this resource wrt training and fine tuning a deep learning model.

Hope this helps!