# Combining scaling, dimensionality reduction, prediction using sklearn pipeline

I would like to use a sklearn pipeline doing this :

( - ) scale the data ( StandardScaler )

( - ) reduce dimensionality ( PCA )

( - ) make a prediction with GradientBoostingRegressor() and GridSearchCV() ( to get the model with best parameters from grid )

( - ) make a prediction with RandomForestRegressor() and GridSearchCV() ( to get the model with best parameters from grid )

( - ) take the mean of both predictions

but I cannot figure out how to proceed.

Do I have to ( scale & predict ) 2 times or can I scale and then predict 2 times ?

Thks.

The problem seems to be that sklearn pipelines are strictly linear in nature, so you can't fit the two models within the pipeline. You could try ensembling the GBM and RF into a new model object to use; for that, use FeatureUnion:
https://stats.stackexchange.com/questions/139042/ensemble-of-different-kinds-of-regressors-using-scikit-learn-or-any-other-pytho
https://stackoverflow.com/questions/43010914/stack-ensemble-estimators-using-sklearn-pipeline-and-gridsearchcv
I don't see immediately how to take the simple average of the scores rather than fitting a linear model, but it shouldn't be hard; maybe as a custom transformation? Then there's also a minor difference from what you're describing: the cross-validation for hyperparameters would happen for the ensemble, so would seek the best result of the ensemble rather than the two individual models.

• I am reacting to that part of Ben Reiniger answer : The problem seems to be that sklearn pipelines are strictly linear in nature, so you can't fit the two models within the pipeline. I think this link ml-ensemble.com could help. – Fabrice BOUCHAREL May 22 at 12:01