1
$\begingroup$

I'm working on a multiclass classification problem, comparing results from SVM and Random Forest classificators. I would like to use gridsearchCV for hyperparameters tuning and find that AUC is the most used metrics for this kind of problem.

I know how to use other metrics of scoring like accuracy etc. but the default ROC_AUC only works for binary class. Is there a method to use AUC in gridsearchCV for multiclass problems?

$\endgroup$
0
$\begingroup$

sklearn's roc_auc_score actually does handle multiclass and multilabel problems, with its average and multiclass parameters. The default average='macro' is fine, though you should consider the alternative(s). But the default multiclass='raise' will need to be overridden. To use that in a GridSearchCV, you can curry the function, e.g.

import functools

multiclass_roc_auc = functools.partial(roc_auc_score, multiclass='ovr')
search = GridSearchCV(estimator=...,
                      eval=multiclass_roc_auc,
                      ...)
| improve this answer | |
$\endgroup$
  • $\begingroup$ I fixed my problem by defining my multiclass_roc_auc function. Your solution is very helpful and slightly shorter, thanks a lot! $\endgroup$ – okraw May 11 at 15:29
1
$\begingroup$

Metrics are independent from ML algorithms, so it doesn't matter which algorithms did you use.

To calculate multiclass AUC you could use lib pRoc in R or use code this link(in Python).

Sources:

| improve this answer | |
$\endgroup$
  • $\begingroup$ I mean the evaluation metrics that the GridsearchCV function uses to find best params $\endgroup$ – okraw Apr 4 at 18:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.