How to build single pipeline with multiple estimators supporting fit and predict?

Question

I have a ML problem where I want to divide the prediction task into subproblems (where I believe specialized models will do better). All these predictions tasks operate independently and will use the same input data - but will have different estimators/targets.

For example:

• single dataset (A)
• shared transformations A -> B
• estimator #1: random forests with target Y1
• estimator #2: GBM classifier with target Y2
• estimator #3: logistic regression with target Y2
• the predictions of each of these models will be output as a tuple (#1, #2, #3)

I'm looking for a simple (or best practice way) to define the above pipeline and train it and be able to use it for prediction. I have looked at sklearn Pipeline but best I can tell you can't use that to have multiple estimators for training/predictions (would love to learn I'm wrong on this).

My fallback option is to build a class that supports fit and predict_proba but under the hood just calls these models sequentially (training in sequence & generating predictions in sequence before returning the tuple of results).

Is there a better way to go about this problem?

Answer 1

Scikit-learn pipelines are designed to chain operations, they are not designed to handle conditional logic.

Your problem is better handled in Python-based logic. Something like:

from sklearn.ensemble      import GradientBoostingClassifier, RandomForestClassifier
from sklearn.linear_model  import LogisticRegression
from sklearn.pipeline      import make_pipeline
from sklearn.preprocessing import StandardScaler


pipe_1 = make_pipeline(StandardScaler(), RandomForestClassifier())
pipe_2 = make_pipeline(StandardScaler(), GradientBoostingClassifier())
pipe_3 = make_pipeline(StandardScaler(), LogisticRegression())

pipe_1.fit(X, y1)
pipe_2.fit(X, y2)
pipe_3.fit(X, y2)

predictions = (pipe_1.predict(X), pipe_2.predict(X), pipe_3.predict(X))