What is the difference between these two curves: learning_curve and validation_curve ?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
9
Both curves show the training and validation scores of an estimator on the y-axis.
A learning curve plots the score over varying numbers of training samples, while a validation curve plots the score over a varying hyper parameter.
The learning curve is a tool for finding out if an estimator would benefit from more data, or if the model is too simple (biased).
Above example shows the training curve for a classifier where training and validation scores converge to a low value. This classifier would hardly benefit from adding more training data; a more expressive model may be more appropriate.
The validation curve is a tool for finding good hyper parameter settings. Some hyper parameters (number of neurons in a neural network, maximum tree depth in a decision tree, amount of regularization, etc.) control the complexity of a model. We want the model to be complex enough to capture relevant information in the training data but not too complex to avoid overfitting.
Above example shows the validation curve over a support vector machine's gamma parameter. A too low value of gamma restricts the model too much; both, training and validation scores are very low. A high value of gamma causes overfitting: very good training score but low validation score. The optimal value is somewhere in the middle, where the curves do not diverge too much.
Image source: scikit-learn documentation
-
$\begingroup$ Thank you, so in principle they are both the same, just with a different parameter on the x-axis...? $\endgroup$– BenOct 28, 2019 at 13:31
-
1$\begingroup$ @Ben In principle, yes. You could even consider the number of training samples a hyper parameter (although more is usually (always?) better). $\endgroup$– MB-FOct 28, 2019 at 13:45
-
$\begingroup$ alright, thanks. I wonder why there are two different functions in sklearn then.. this confused me heavily. However, things move on :) Thx! $\endgroup$– BenOct 28, 2019 at 13:48
-
$\begingroup$ I have another question: What about the loss curve? (How) do I make use out of it? And in a regression task I do not have accuracy, so I only plot the loss curve? $\endgroup$– BenOct 29, 2019 at 10:54
-
$\begingroup$ You use the loss curve to select a good value for the parameter. Both curves are general over estimator type: score can be accuracy, r2-score, mean-squared-error, f1-score, whatever you choose... $\endgroup$– MB-FOct 29, 2019 at 12:00