3
$\begingroup$

What is the difference between these two curves: learning_curve and validation_curve ?

$\endgroup$
4
$\begingroup$

Both curves show the training and validation scores of an estimator on the y-axis.

A learning curve plots the score over varying numbers of training samples, while a validation curve plots the score over a varying hyper parameter.

The learning curve is a tool for finding out if an estimator would benefit from more data, or if the model is too simple (biased).

enter image description here

Above example shows the training curve for a classifier where training and validation scores converge to a low value. This classifier would hardly benefit from adding more training data; a more expressive model may be more appropriate.

The validation curve is a tool for finding good hyper parameter settings. Some hyper parameters (number of neurons in a neural network, maximum tree depth in a decision tree, amount of regularization, etc.) control the complexity of a model. We want the model to be complex enough to capture relevant information in the training data but not too complex to avoid overfitting.

enter image description here

Above example shows the validation curve over a support vector machine's gamma parameter. A too low value of gamma restricts the model too much; both, training and validation scores are very low. A high value of gamma causes overfitting: very good training score but low validation score. The optimal value is somewhere in the middle, where the curves do not diverge too much.

Image source: scikit-learn documentation

$\endgroup$
  • $\begingroup$ Thank you, so in principle they are both the same, just with a different parameter on the x-axis...? $\endgroup$ – Ben Oct 28 at 13:31
  • 1
    $\begingroup$ @Ben In principle, yes. You could even consider the number of training samples a hyper parameter (although more is usually (always?) better). $\endgroup$ – kazemakase Oct 28 at 13:45
  • $\begingroup$ alright, thanks. I wonder why there are two different functions in sklearn then.. this confused me heavily. However, things move on :) Thx! $\endgroup$ – Ben Oct 28 at 13:48
  • $\begingroup$ I have another question: What about the loss curve? (How) do I make use out of it? And in a regression task I do not have accuracy, so I only plot the loss curve? $\endgroup$ – Ben Oct 29 at 10:54
  • $\begingroup$ You use the loss curve to select a good value for the parameter. Both curves are general over estimator type: score can be accuracy, r2-score, mean-squared-error, f1-score, whatever you choose... $\endgroup$ – kazemakase Oct 29 at 12:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.