3
$\begingroup$

In random forests, where our estimators are decision trees, we do column (feature) sampling without replacement within an estimator, and with replacement in between estimators. This is perfectly fine as we are trying to reduce the high variance of individual decision trees.

But what is the need to do row sampling?

Usually more the data, the better it is for a model to learn, and even if i dont have any computational resource limitation, why do we have to do row sampling in estimators for random forest classifier?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
2
$\begingroup$

First I think your understanding of “column sampling” is incorrect. Random forest try’s a subset of features for each split. It does not sample without replacement within an individual tree. Random forest samples rows with replacement (bootstrap samples) to remove correlation between the decision trees. Think about it, if you didn’t do this even though you create each split based on only a subset of features, your trees would end up looking fairly similar (or at least more similar than if you bootstrapped). You do have larger bias due to creating trees based only on about ~63% unique values but the decrease in variance by having more uncorrelated trees makes up for it.

Improve this answer
$\endgroup$
0
$\begingroup$

I think it is a way to reduce bias. If you're training a Random Forest with 100 trees, then you will grow these trees with (potentially) 100 different training sets. You can achieve the "wisdom of the crowds" since there is a crowd formed by these training sets.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I think "wisdom of the crowds" analogy is problematic, since it refers to the benefit of larger number of individuals. But row sampling is about the benefit of smaller number of individuals, given the number of trees (weak learners) is fixed. $\endgroup$
    – Esmailian
    Mar 25, 2019 at 21:20
  • $\begingroup$ Here the larger number of individuals is due to having different training set at each iteration. That's what I mean for "the crowd". $\endgroup$
    – Matteo Felici
    Mar 25, 2019 at 21:38
0
$\begingroup$

We do row sampling beacuse, for each tree in Random Forest we have different training set,and thus each tree is different in its prediction capabilities and that would make a rich forest

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.