6
$\begingroup$

Learning about random forest for the first time and I'm not clear if at each level, if we've already used a feature to make a decision, whether we can use the same feature again at further levels.

$\endgroup$
0

2 Answers 2

8
$\begingroup$

If the feature is discrete/categorial/binary, it won't use the already used feature to split at further levels in any given branch of the tree. But key thing to note here is that the same feature can be used in a different branch of the tree at a different level.

Decision Tree

In the above diagram, we can see that same 'size' feature has been used at two levels 'level 1' and 'level 2', but in different branches of the tree.

On the other hand, if the variable is a continuous value, it uses threshold splits at each level and in this case, same feature can be used multiple times in any given branch of the decision tree.

Since random forests is an ensemble of decision trees, the same theory applies to Random Forests as well.

Reference: http://www.cse.msu.edu/~cse802/DecisionTrees.pdf

$\endgroup$
2
  • 5
    $\begingroup$ All standard implementations of random forests use binary splits. There, any feature can be used multiple times in a tree as long as it still qualifies for a split.Thus said, a binary feature $X$ is used only once because after the split, its values are constant 0 or 1 within either branch. $\endgroup$
    – Michael M
    Jul 23, 2017 at 13:44
  • $\begingroup$ Agreed with @MichaelM, from a lot of the research I've done, it seems like for individual trees in random forests, it's possible to re-use features within the same branch, but not for a plain decision tree $\endgroup$ Aug 8, 2019 at 17:12
0
$\begingroup$

Very similar to: https://stats.stackexchange.com/questions/452697/are-features-in-random-forest-sampled-with-replacement

The sampling without replacement is only done at the node level (to ensure you choose n unique features). There is no condition that stops you from reusing features in the random selection pool, even if a feature was already used for a split.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.