My input data is a (23948,) pandas.Series of strings containing newspaper headlines. My target are 20 labels of the headline (e.g. 'crime', 'politics') each binarily encoded with [0, 1]. The labels are not exclusive, a headline could be about crime and politics at the same time. I would like to compare three algorithms for this problem. I use the following pipeline to predict the labels:

pipeline = Pipeline([
    ('vect', CountVectorizer(tokenizer=tokenize)),
    ('tfidf', TfidfTransformer()),
    ('clf', MultiOutputClassifier(RandomForestClassifier()))

parameters = [
    {"clf": [RandomForestClassifier()],
     "clf__n_estimators": [10, 100, 250],
    {"clf": [LinearSVC()],
     "clf__C": [1.0, 10.0, 100.0, 1000.0],
    {"clf": [GaussianNB()]}

rkf = RepeatedKFold(

cv = GridSearchCV(

The pipeline works fine for the random forest, but breaks at the LinearSVC() with the following error:

ValueError: bad input shape (20972, 20)

If I remove the LinearSVC(), it stops at:

A sparse matrix was passed, but dense data is required. Use X.toarray() to convert to a dense numpy array.

There is a list "Support multilabel:" in the sci-kit learn documentation on multilabel classification (https://scikit-learn.org/stable/modules/multiclass.html) and only the random forest is included. However, the documentation states that "Multioutput classification support can be added to any classifier with MultiOutputClassifier."

I am bit confused, do LinearSVC() and GaussianNB() support multilabel classification when wrapped in MultiOutputClassifier()? If not, is there a workaround?


1 Answer 1


When you run your grid search, the clf step of the pipeline is replaced by each of RandomForestClassifier, LinearSVC, GaussianNB; you never actually use the MultiOutputClassifier.

You should be able to just wrap the two offending classifiers with a MultiOutputClassifier. You'll need to prefix your hyperparameters with estimator__ to get through the MOC into the underlying classifier: clf__estimator__C.

  • $\begingroup$ Thanks, this works. After reading this post stackoverflow.com/questions/42819460/… I am wondering whether OneVsRestClassifier() does actually make more sense. Is the only difference that in MultiOutputClassifier the variables do not need to be binary but may feature multiple classes? $\endgroup$ Commented Aug 29, 2019 at 15:46
  • 1
    $\begingroup$ @00schneider, "The labels are not exclusive" suggests that one-vs-rest is not appropriate for you, yes? $\endgroup$
    – Ben Reiniger
    Commented Aug 29, 2019 at 16:19
  • $\begingroup$ Hm, I guess I am making a follow up post... $\endgroup$ Commented Aug 30, 2019 at 8:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.