How to get mean test scores from GridSearchCV with multiple scorers - scikit-learn

Question

I'm trying to get mean test scores from scikit-learn's GridSearchCV with multiple scorers.

grid.cv_results_ displays lots of info. But grid.cv_results_['mean_test_score'] keeps giving me an error.

I've checked the docs and similar questions with no luck. What am I doing wrong?

Code:

scorers = {
    'r2': 'r2',
    'nmsle': 'neg_mean_squared_log_error',
    'nmse': 'neg_mean_squared_error'
}

params = [
    {
        'regressor': [GradientBoostingRegressor()], 'preprocessing': [None],
    },
]

grid = GridSearchCV(pipe, params, cv=5, scoring=scorers, refit='nmse')
grid.fit(X_t, y_train)

grid.cv_results_['mean_test_score']

Error:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-161-a021fe7b05bb> in <module>
     20 print(f'Val score set score rmse: {rm}')
     21 
---> 22 grid.cv_results_['mean_test_score']
     23 
     24 # for mean in means:

~/anaconda3/lib/python3.6/site-packages/sklearn/utils/deprecation.py in __getitem__(self, key)
    124             warn_args, warn_kwargs = self._deprecations[key]
    125             warnings.warn(*warn_args, **warn_kwargs)
--> 126         return super(DeprecationDict, self).__getitem__(key)
    127 
    128     def get(self, key, default=None):

KeyError: 'mean_test_score'
```

Answer 1

For multi-metric evaluation, the scores for all the scorers are available in the cv_results_ dict at the keys ending with that scorer's name ('_scorer_name'). so use

grid.cv_results_['mean_test_(scorer_name)']

Ex: grid.cv_results_['mean_test_r2']