# Validation and training loss of a model are not stable

Below I have a model trained and the loss of both the training dataset (blue) and validation dataset (orange) are shown. From my understanding, the ideal case is that both validation and training loss should converge and stabilize in order to tell that the model does not underfit or overfit. But I am not sure about the model below. What you can tell from it's loss, please?

In addition, this is the accuracy of the model:

Update: after setting learning rate to 0.0001 as the answer suggested, I got the following loss:

And accuracy of the model:

## 1 Answer

It seems like you're over fitting. There are tones of articles and blogs on how to avoid over fitting, but I mention some of them here anyway:

• Reduce your learning rate to a very small number like 0.001 or even 0.0001.

• Provide more data.

• Set Dropout rates to a number like 0.2. Keep them uniform across the network. Another modern approach is the idea of using Batch Normalization instead of Dropout.

• Try decreasing the batch size.

• Using appropriate optimizer: You may need to experiment a bit on this. Use different optimizer on the same network, and select an optimizer which gives you the least loss.

• Also try reducing the size of your model.

• Thank you for your response. Can you please explain why the model overfits? I will try the solutions meanwhile.
– Avv
Dec 2, 2022 at 20:58
• I added the new loss and accuracy of the model after setting learning rate to $0.0001$
– Avv
Dec 2, 2022 at 21:14
• The fluctuations have disappeared which is a good sign. I'm afraid that the reason your accuracy isn't getting better is because that you need more data, or maybe even try some preprocessing the data. @Avv Dec 3, 2022 at 12:18
• Thank you. But why you said first we have overfitting before from the first 2 figures?
– Avv
Dec 3, 2022 at 20:20