GridSearch mean_test_score vs mean_train_score

Question

I am working with scikit learn and GridSearch in order to find the best parameters in my classifiers.

I have a map of different hyperparameters and I want to print out GridSearch results, but I do not understand one thing - what is the difference between mean_test_score and mean_train_score?

As I understand, GridSearch performs cross-validation in order to find the best classifier, but how do these 2 params differ from one another? I always thought that cross-validation gives only one mean, which is a mean of the performance from trained models using N subsets of given data. For example, if I perform a cross-validation with X subsets, I will have X different accuracy scores and then I will have only one mean value.

So, how can I interpret these 2 parameters and what is the difference between them?

Answer 1

When you do k-fold cross-validation, you train k models, each one of them leaving the proportion $1/k$ of the data out. For each of the models, you can compute its train error and validation error. The train error will be the error on the data selected to train the model, and the validation error will be the data left out of the training.

For this reason, you have k training errors and k validation/test errors, and computing their averages will give you the quantities you are talking about.