2
$\begingroup$

Below I have a model trained and the loss of both the training dataset (blue) and validation dataset (orange) are shown. From my understanding, the ideal case is that both validation and training loss should converge and stabilize in order to tell that the model does not underfit or overfit. But I am not sure about the model below. What you can tell from it's loss, please?

enter image description here

In addition, this is the accuracy of the model:

enter image description here

Update: after setting learning rate to 0.0001 as the answer suggested, I got the following loss:

enter image description here

And accuracy of the model:

enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

It seems like you're over fitting. There are tones of articles and blogs on how to avoid over fitting, but I mention some of them here anyway:

  • Reduce your learning rate to a very small number like 0.001 or even 0.0001.

  • Provide more data.

  • Set Dropout rates to a number like 0.2. Keep them uniform across the network. Another modern approach is the idea of using Batch Normalization instead of Dropout.

  • Try decreasing the batch size.

  • Using appropriate optimizer: You may need to experiment a bit on this. Use different optimizer on the same network, and select an optimizer which gives you the least loss.

  • Also try reducing the size of your model.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you for your response. Can you please explain why the model overfits? I will try the solutions meanwhile. $\endgroup$
    – Avv
    Dec 2, 2022 at 20:58
  • $\begingroup$ I added the new loss and accuracy of the model after setting learning rate to $0.0001$ $\endgroup$
    – Avv
    Dec 2, 2022 at 21:14
  • 1
    $\begingroup$ The fluctuations have disappeared which is a good sign. I'm afraid that the reason your accuracy isn't getting better is because that you need more data, or maybe even try some preprocessing the data. @Avv $\endgroup$
    – Amirhossein Rezaei
    Dec 3, 2022 at 12:18
  • $\begingroup$ Thank you. But why you said first we have overfitting before from the first 2 figures? $\endgroup$
    – Avv
    Dec 3, 2022 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.