1
$\begingroup$

My understanding of boosting is just training models sequentially and learning from its previous mistakes.

Can boosting algorithms be built with bunch of logistic regression? or logistic regression + decision trees?

If yes, I would like to know some papers or books that covers this topic in-depth.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Boosting is not limited to tree-based models. Find some more information here:

P. Bühlmann, T. Hothorn (2007), "Boosting Algorithms: Regularization, Prediction and Model Fitting", Statistical Science 22(4), p. 477-505.

I implemented L2 linear regression boosting from Section 3.3 (p. 483) from the paper above in this R-code. You may replace the L2 model by a logit model and see how it works.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Perfect! Will look into it, thanks! $\endgroup$
    – haneulkim
    Commented Sep 8, 2021 at 10:27
  • 1
    $\begingroup$ @haneulkim BTW: the idea of combining estimators in a boosting estimator sounds interesting. IMO this would require averaging or stacking. I do not know if there is reasearch on this issue and/or if it brings an additional advantage. Would be happy to know if you gain some insights on this. Cheers… $\endgroup$
    – Peter
    Commented Sep 8, 2021 at 21:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.