In boosting process we get a better accuracy for training data, but there is a lot of chance to over fit. For bagging ensemble method over fitting chance is lower than boosting. Why we use boosting and for what condition ??


1 Answer 1


Bagging (and features sampling) aim to reduce variance by providing low-correlated trees. Estimators can then be aggregated together to reduce variance. Reason is simple decision trees tend to quickly overfit in the bottom nodes. Bagging and features sampling only adress high-variance (overfitting) problems.

On the other hand, boosting is a meta-algorithm that aims to reduce both bias and variance by training estimators in a sequential way. Each predictor tries to reduce residuals from previous estimators.

In my opinion, the two meta-algorithms can't be easily compared that way. They will give you different results depending of data distributions, potential outliers, usecase... Potentially, boosting is more sensitive to outliers than bagging for example. I think trying the 2 methods is your best chance and you don't have to restrict to only one model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.