2
$\begingroup$

I a trying to understand this learning curve of a classification problem. But I am not sure what to infer. I believe that I have overfitting but I cannot sure.

  1. Very low training loss that’s very slightly increasing upon adding training examples.
  2. "Gradually decreasing validation loss (without flattening) upon adding training examples". However, I do not see any gap at the end of the lines something that is usually can be found in an overfitting model

On the other hand, I might have underfitting as:

  1. Learning curve of an underfit model has a low training loss at the beginning which gradually increases upon adding training examples and stay flat, indicating the addition of more training examples can’t improve the model performance on unseen data
  2. Training loss and validation loss are close to each other at the end

However, the train error is not to big something that usually is found on underfitting models

I am confused Can you please provide me with some advice?

enter image description here

enter image description here

New contributor
xavi is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
1
$\begingroup$

Since you didn't mention this in the description, let me first emphasize that these graphs show the performance for different sizes of the training set, i.e. this is the result of an ablation study. This is important because it means that every point on the X axis represents a different model.

Now, what these two graphs show is clear overfitting on the left part of the graphs: the performance is very high on the training set and very low on the test/validation set. So all the models which are trained with less than around 450-500 instances are overfit. But the two curves converge up to the point where the training and validation performance are equal, so the last model with around 500 instances does not have any sign of overfitting.

Underfitting is less common and less easy to detect from this kind of curve, but it would be characterized by a low performance even on the training set. This does not happen here, so I think this can be ruled out. The fact that performance is the same on the training and test set only shows that there's no overfitting, not that the model is underfit.

$\endgroup$
4
  • $\begingroup$ ok, thank you. So as in the beginning, I have overfitting and I end up with no overfitting what does this means for me. Will I take action towards handling overfitting or not? $\endgroup$
    – xavi
    Jan 12 at 23:35
  • $\begingroup$ @xavi as I said in the answer, these graphs show the performance for multiple models trained on different amount of instances. So the only action needed to avoid overfitting is to use a model trained with at least 450-500 instances. $\endgroup$
    – Erwan
    Jan 13 at 11:28
  • $\begingroup$ @Ervan Thank you. I used 10 fold cross-validation. When do you say multiple models are you referring to these models generated from the CV? $\endgroup$
    – xavi
    Jan 13 at 11:52
  • $\begingroup$ @xavi no, you don't only use CV here, because CV doesn't give results for different sizes of training set. There's a misunderstanding about what kind of experiment these graphs represent, but I don't even know which algorithm you're using. Could you share the code that you're using, or at least give some details about it? The other explanation would be that there's a mistake in the labels of the graphs and in this case my interpretation is wrong. $\endgroup$
    – Erwan
    Jan 13 at 12:07

Your Answer

xavi is a new contributor. Be nice, and check out our Code of Conduct.

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.