Can anyone explain me the basic difference between bagging and boosting and which technique can be used in which scenario?
2
-
$\begingroup$ You need to know how these sampling techniques work or algorithms based on these techniques? $\endgroup$– CodeMaster GoGoCommented Oct 12, 2018 at 9:39
-
$\begingroup$ quora.com/Whats-the-difference-between-boosting-and-bagging $\endgroup$– user2974951Commented Oct 12, 2018 at 9:40
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Bagging: Also known as Bootstrap Aggregation is an ensemble method. First, we create random samples of the training data set (sub sets of training data set). Then, we build a classifier for each sample. Finally, results of these multiple classifiers are combined using average or majority voting. Bagging helps to reduce the variance error.
Boosting provides sequential learning of the predictors. The first predictor is learned on the whole data set, while the following are learnt on the training set based on the performance of the previous one. It starts by classifying original data set and giving equal weights to each observation. If classes are predicted incorrectly using the first learner, then it gives higher weight to the missed classified observation. Being an iterative process, it continues to add classifier learner until a limit is reached in the number of models or accuracy. Boosting has shown better predictive accuracy than bagging, but it also tends to over-fit the training data as well.
Algorithms work on these techniques
Bagging: Random Forest
Boosting : Ada Boost, Gradient Boosting, XGBoosting, etc
answered Oct 12, 2018 at 9:51