How to train multioutput classification with hyperparameter tuning in sklearn?

Question

I am working on a simple multioutput classification problem and noticed this error showing up whenever running the below code:

ValueError: Target is multilabel-indicator but average='binary'. Please 
choose another average setting, one of [None, 'micro', 'macro', 'weighted', 'samples'].

I understand the problem it is referencing, i.e., when evaluating multilabel models one needs to explicitly set the type of averaging. Nevertheless, I am unable to figure out where this average argument should go to; accuracy_score, precision_score, recall_score built-in methods have this argument which I do not use explicitly in my code. MultiOutputClassifier doesn't have such an argument, neither does the RandomizedSearchCV's .fit() method. I also tried passing methods like precision_score(average='micro') directly to the scoring and refit arguments of RandomizedSearchCV but that didn't solve it either since methods such as precision_score() require correct and true y labels as arguments, which I have no access to in the individual K-folds of the randomized search.

Full code with data:

from sklearn.datasets import make_multilabel_classification
from sklearn.naive_bayes import MultinomialNB
from sklearn.multioutput import MultiOutputClassifier
from sklearn.model_selection import RandomizedSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import MinMaxScaler

X, Y = make_multilabel_classification(
    n_samples=1000,
    n_features=2,
    n_classes=5,
    n_labels=2
)

pipe = Pipeline(
    steps = [
        ('scaler', MinMaxScaler()),
        ('model', MultiOutputClassifier(MultinomialNB()))
    ]
)

search = RandomizedSearchCV(
    estimator = pipe,
    param_distributions={'model__estimator__alpha': (0.01,1)},
    scoring = ['accuracy', 'precision', 'recall'],
    refit = 'precision',
    cv = 5
).fit(X, Y)
```

Answer 1

The solution to this problem is relatively straight-forward, as the scikit-learn documentation mentions you can use strings (or a list of strings) to specify the scoring method(s) used. Following the link of scoring options you will see a list of predefined values for the scorers. For some scorers there are multiple types as mentioned by the error you are getting, e.g. for the F1 score you can use the default one for binary targets or any of the micro, macro, weighted, or samples options. You therefore simply have to change the strings you are providing to RandomizedSearchCV for the scoring and refit parameters to select the specific type of scorer you want to use. This would look something like this:

search = RandomizedSearchCV(
    estimator=pipe,
    param_distributions={'model__estimator__alpha': (0.01, 1)},
    scoring=['accuracy', 'precision_micro', 'recall_micro'],
    refit='precision_micro',
    cv = 5
).fit(X, Y)