0
$\begingroup$

Does that overfitting ? How can I interpret the curve ?

enter image description here

$\endgroup$
0
$\begingroup$

It looks like overfitting. Check this article to learn about interpreting different types of learning curves. TensorFlow also has a tutorial on this topic. There is a clear split between the curves at about epoch 10 where training keeps learning at a much faster rate compared to validation.

But as you point out validation loss stays pretty much stable with very little noise. In the same way, there is very little noise in training loss as well. One reason this might be case is if every batch is a perfect representative of the dataset. Maybe you are using your entire dataset in each iteration? If this is the case, you can try using smaller batch sizes. Another possibility is that the learning rate might be too high. This could also cause the model to converge to get stuck too early. You can either use a smaller learning rate or use callbacks API to lower learning rate when stuck on plateaus.

$\endgroup$
1
  • $\begingroup$ thanks a lot but excuse me .. how the stability of a valid curve after some epochs becomes overfitting .. that is the point i didn't get it .. as I got is the overfitting decrease or increase opposite the training curve $\endgroup$
    – USER
    Oct 8 at 8:17
0
$\begingroup$

Hard to tell. Usually you would expect some difference between the two, and you would worry if they have dissimilar shapes. But yours are very similar, and the validation curve has a smaller loss from the start, compared to the training loss. Maybe the training/validation split was just unfortunate. Try to train the model again with a new validation sample, see if the pattern persists.

$\endgroup$
3
  • $\begingroup$ Thanks for replying . i read and found this training loss that shows improvement and similarly a learning curve for validation loss that shows improvement, but a large gap remains between both curves. does that mean i need to increase the size of training dataset .. my dataset now is 30.000 and testing is 5000 $\endgroup$
    – USER
    Oct 8 at 9:11
  • $\begingroup$ @user2974951 I think the reason validation curve has a smaller loss in epoch 1 is probably because training loss is measured during the epoch and validation loss is measured after the epoch - when the updates of the first epoch are completed. $\endgroup$
    – serali
    Oct 8 at 9:26
  • $\begingroup$ @serali I would try to increase the test set, or just try another random set. I don't know what kind of data you have and/or if there is dependence between them, so a couple more runs of random shuffling might show a different picture. If not, then maybe you need to tune your model. $\endgroup$ Oct 13 at 5:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.