1
$\begingroup$

My understanding of boosting is just training models sequentially and learning from its previous mistakes.

Can boosting algorithms be built with bunch of logistic regression? or logistic regression + decision trees?

If yes, I would like to know some papers or books that covers this topic in-depth.

$\endgroup$
2
$\begingroup$

Boosting is not limited to tree-based models. Find some more information here:

P. Bühlmann, T. Hothorn (2007), "Boosting Algorithms: Regularization, Prediction and Model Fitting", Statistical Science 22(4), p. 477-505.

I implemented L2 linear regression boosting from Section 3.3 (p. 483) from the paper above in this R-code. You may replace the L2 model by a logit model and see how it works.

$\endgroup$
2
  • $\begingroup$ Perfect! Will look into it, thanks! $\endgroup$
    – haneulkim
    Sep 8 '21 at 10:27
  • 1
    $\begingroup$ @haneulkim BTW: the idea of combining estimators in a boosting estimator sounds interesting. IMO this would require averaging or stacking. I do not know if there is reasearch on this issue and/or if it brings an additional advantage. Would be happy to know if you gain some insights on this. Cheers… $\endgroup$
    – Peter
    Sep 8 '21 at 21:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.