-2
$\begingroup$

I am applying a active learning with a SGDClassifier (log loss function) as the base learner on some data and I have the following graphs representing the learning curve of queries vs error rate. The green is the validation error and blue is the training error.

Is my model overfitting or has high variance in both graphs?

enter image description here

enter image description here

$\endgroup$
6
  • $\begingroup$ Looks like its beginning to overfit to me $\endgroup$
    – zacdav
    Apr 6 '18 at 4:50
  • $\begingroup$ Im little confuse though, I thought that the training and test error starts a bit higher in the graphs and then begin to decrease. But in this case it starts lower in the graphs and increases. Is this right ? $\endgroup$
    – DPascal
    Apr 6 '18 at 5:28
  • $\begingroup$ Well consider the moment where both graphs have the training and testing sets diverge, this is telling you that your validation error isnt improving, however, your training set is getting better. You'd be wasting time continuing the process without making changes elsewhere to evaluate or try $\endgroup$
    – zacdav
    Apr 6 '18 at 5:46
  • $\begingroup$ Ahh I see. What if I try to update the model with new observations with a SGDCLASSIFER in a online environment ? I imagine that would still be relevant. $\endgroup$
    – DPascal
    Apr 6 '18 at 5:55
  • $\begingroup$ @DPascal What online environment?? Don't add new observations unless you know what you're doing. $\endgroup$
    – SmallChess
    Apr 6 '18 at 5:57
2
$\begingroup$

Overfitting looks more likely because:

  • After some queries your validation errors are systematically higher than training errors, which is probably not what you want.
  • After some queries, your training errors slowly fall while validation errors remain constant. It's like your classifier is memorising your data set at a slow pace.

When the number of queries is small, it looks like your classifier did better simply because it was a simpler data set. Again, overfitting can also be an issue.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.