I have a ML problem where I want to divide the prediction task into subproblems (where I believe specialized models will do better). All these predictions tasks operate independently and will use the same input data - but will have different estimators/targets.

For example:

  • single dataset (A)
  • shared transformations A -> B
  • estimator #1: random forests with target Y1
  • estimator #2: GBM classifier with target Y2
  • estimator #3: logistic regression with target Y2
  • the predictions of each of these models will be output as a tuple (#1, #2, #3)

I'm looking for a simple (or best practice way) to define the above pipeline and train it and be able to use it for prediction. I have looked at sklearn Pipeline but best I can tell you can't use that to have multiple estimators for training/predictions (would love to learn I'm wrong on this).

My fallback option is to build a class that supports fit and predict_proba but under the hood just calls these models sequentially (training in sequence & generating predictions in sequence before returning the tuple of results).

Is there a better way to go about this problem?


1 Answer 1


Scikit-learn pipelines are designed to chain operations, they are not designed to handle conditional logic.

Your problem is better handled in Python-based logic. Something like:

from sklearn.ensemble      import GradientBoostingClassifier, RandomForestClassifier
from sklearn.linear_model  import LogisticRegression
from sklearn.pipeline      import make_pipeline
from sklearn.preprocessing import StandardScaler

pipe_1 = make_pipeline(StandardScaler(), RandomForestClassifier())
pipe_2 = make_pipeline(StandardScaler(), GradientBoostingClassifier())
pipe_3 = make_pipeline(StandardScaler(), LogisticRegression())

pipe_1.fit(X, y1)
pipe_2.fit(X, y2)
pipe_3.fit(X, y2)

predictions = (pipe_1.predict(X), pipe_2.predict(X), pipe_3.predict(X))
  • $\begingroup$ Thanks Brian - I was hoping I could feed MultiOutputClassifier a pipeline that contained a chain of operations - eg. normalization > feature selection > classifier. Do you know if that is possible - such that your example above could be simplified & optimized with the use of Parallel within MultiOutputClassifier? $\endgroup$
    – Mark
    Commented Aug 3, 2022 at 8:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.