When implementing stacking for model building and prediction (For example using sklearn's StackingRegressor function) what is the appropriate choice of models for the base models and final meta model?

Should weak/linear models be used as the base models and an ensemble model as the final meta model (For example: Lasso, Ridge and ElasticNet as base models, and XGBoost as a meta model). Or should non-linear/ensemble models be used as base models and linear regression as the final meta model (For example, XGBoot, Random Forest, LightGBM as the base models, and Ridge as the final meta model)

  • 2
    $\begingroup$ I think this depends on the data and problem. However, it is important to have "diverse" models as base learners. So "weak" linear models may be okay as well as "diverse" boosted or tree based models (e.g. different objectives in boosting or so). The choice of the meta learner still depends on the structure of the data after stacking. You could check some quick linear models first. $\endgroup$ – Peter Sep 9 '20 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.