4
$\begingroup$

I'm trying to get mean test scores from scikit-learn's GridSearchCV with multiple scorers.

grid.cv_results_ displays lots of info. But grid.cv_results_['mean_test_score'] keeps giving me an error.

I've checked the docs and similar questions with no luck. What am I doing wrong?

Code:

scorers = {
    'r2': 'r2',
    'nmsle': 'neg_mean_squared_log_error',
    'nmse': 'neg_mean_squared_error'
}

params = [
    {
        'regressor': [GradientBoostingRegressor()], 'preprocessing': [None],
    },
]

grid = GridSearchCV(pipe, params, cv=5, scoring=scorers, refit='nmse')
grid.fit(X_t, y_train)

grid.cv_results_['mean_test_score']

Error:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-161-a021fe7b05bb> in <module>
     20 print(f'Val score set score rmse: {rm}')
     21 
---> 22 grid.cv_results_['mean_test_score']
     23 
     24 # for mean in means:

~/anaconda3/lib/python3.6/site-packages/sklearn/utils/deprecation.py in __getitem__(self, key)
    124             warn_args, warn_kwargs = self._deprecations[key]
    125             warnings.warn(*warn_args, **warn_kwargs)
--> 126         return super(DeprecationDict, self).__getitem__(key)
    127 
    128     def get(self, key, default=None):

KeyError: 'mean_test_score'
```
$\endgroup$

1 Answer 1

4
$\begingroup$

For multi-metric evaluation, the scores for all the scorers are available in the cv_results_ dict at the keys ending with that scorer's name ('_scorer_name'). so use

grid.cv_results_['mean_test_(scorer_name)']

Ex: grid.cv_results_['mean_test_r2']

$\endgroup$
3
  • $\begingroup$ Thanks! That's easy. Where did you find the answer? $\endgroup$
    – jeffhale
    Jan 10, 2019 at 20:01
  • 1
    $\begingroup$ scikit learn documentation $\endgroup$
    – Uday
    Jan 10, 2019 at 20:15
  • $\begingroup$ I am getting a key error trying this. I don't get why $\endgroup$
    – Atirag
    Feb 5, 2021 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.