Learning about random forest for the first time and I'm not clear if at each level, if we've already used a feature to make a decision, whether we can use the same feature again at further levels.
$\begingroup$
$\endgroup$
0
asked Mar 14, 2016 at 23:38
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
If the feature is discrete/categorial/binary, it won't use the already used feature to split at further levels in any given branch of the tree. But key thing to note here is that the same feature can be used in a different branch of the tree at a different level.
In the above diagram, we can see that same 'size' feature has been used at two levels 'level 1' and 'level 2', but in different branches of the tree.
On the other hand, if the variable is a continuous value, it uses threshold splits at each level and in this case, same feature can be used multiple times in any given branch of the decision tree.
Since random forests is an ensemble of decision trees, the same theory applies to Random Forests as well.
-
5$\begingroup$ All standard implementations of random forests use binary splits. There, any feature can be used multiple times in a tree as long as it still qualifies for a split.Thus said, a binary feature $X$ is used only once because after the split, its values are constant 0 or 1 within either branch. $\endgroup$ Commented Jul 23, 2017 at 13:44
-
$\begingroup$ Agreed with @MichaelM, from a lot of the research I've done, it seems like for individual trees in random forests, it's possible to re-use features within the same branch, but not for a plain decision tree $\endgroup$ Commented Aug 8, 2019 at 17:12
$\begingroup$
$\endgroup$
Very similar to: https://stats.stackexchange.com/questions/452697/are-features-in-random-forest-sampled-with-replacement
The sampling without replacement is only done at the node level (to ensure you choose n unique features). There is no condition that stops you from reusing features in the random selection pool, even if a feature was already used for a split.
