In boosting process we get a better accuracy for training data, but there is a lot of chance to over fit. For bagging ensemble method over fitting chance is lower than boosting. Why we use boosting and for what condition ??
$\begingroup$
$\endgroup$
asked Feb 22, 2018 at 10:09
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Bagging (and features sampling) aim to reduce variance by providing low-correlated trees. Estimators can then be aggregated together to reduce variance. Reason is simple decision trees tend to quickly overfit in the bottom nodes. Bagging and features sampling only adress high-variance (overfitting) problems.
On the other hand, boosting is a meta-algorithm that aims to reduce both bias and variance by training estimators in a sequential way. Each predictor tries to reduce residuals from previous estimators.
In my opinion, the two meta-algorithms can't be easily compared that way. They will give you different results depending of data distributions, potential outliers, usecase... Potentially, boosting is more sensitive to outliers than bagging for example. I think trying the 2 methods is your best chance and you don't have to restrict to only one model.
