0
$\begingroup$

I am working with scikit learn and GridSearch in order to find the best parameters in my classifiers.

I have a map of different hyperparameters and I want to print out GridSearch results, but I do not understand one thing - what is the difference between mean_test_score and mean_train_score?

As I understand, GridSearch performs cross-validation in order to find the best classifier, but how do these 2 params differ from one another? I always thought that cross-validation gives only one mean, which is a mean of the performance from trained models using N subsets of given data. For example, if I perform a cross-validation with X subsets, I will have X different accuracy scores and then I will have only one mean value.

So, how can I interpret these 2 parameters and what is the difference between them?

$\endgroup$
2
$\begingroup$

When you do k-fold cross-validation, you train k models, each one of them leaving the proportion $1/k$ of the data out. For each of the models, you can compute its train error and validation error. The train error will be the error on the data selected to train the model, and the validation error will be the data left out of the training.

For this reason, you have k training errors and k validation/test errors, and computing their averages will give you the quantities you are talking about.

$\endgroup$
  • 1
    $\begingroup$ Could you be more precise about train error? I don't get it - if you train model based on some data and then you use this set of data to test model, you should have 100% accurate model - right? $\endgroup$ – TheOpti May 7 '18 at 12:24
  • 1
    $\begingroup$ No, in general training accuracy is lower than 100%. Take logistic regression, unless your data are linearly separable your training accuracy will be lower than 100%. $\endgroup$ – David Masip May 7 '18 at 13:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.