2
$\begingroup$

This is a citation from "Hands-on machine learning with Scikit-Learn, Keras and TensorFlow" by Aurelien Geron:

"Bootstrapping introduces a bit more diversity in the subsets that each predictor is trained on, so bagging ends up with a slightly higher bias than pasting, but this also means that predictors end up being less correlated so the ensemble’s variance is reduced."

I can't understand why bagging, as compared to pasting, results in higher bias and lower variance. Can anyone provide an intuitive explanation of this?

$\endgroup$
2
$\begingroup$

Let's say we have a set of 40 numbers from 1 to 40. I have to pick 4 subsets of 10 numbers.

Case 1 - Bagging -
I will pick the first 10, then put it back and pick the next 10 again. This will result in some number coming out again and will reduce the variance among the subsets.

Case 2 - Pasting -
Here, all of my set will have 10 distinct values. This means higher variance among the subsets.

But in this case, all subsets are not independent e.g. after 3rd set, the subsets 4th is predictable. This means that the subsets are correlated. This was the other points from Geron.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.