3
$\begingroup$

In Random Forest method, for each tree we randomly select a set of variables (features) of fixed size. But once this set is frozen for that particular tree, does the tree behave like a regular decision tree algorithm?

I am assuming that random forest is nothing but generating a bunch of classical 'decision trees' and taking their votes towards the final classification. But in many places whatever description I have read seems to suggest that; for a given decision tree within the forest even at each node we randomly select variables. Is that the case?

Does it mean that at each node in the tree, we randomly select m variables from the variable set which is fixed for that tree? Or from the global variable set of the training dataset? And then from the selected set of variables we select 1 variable heuristically (e.g. whichever variable maximises information gain) -- is that a correct statement?

$\endgroup$
3
$\begingroup$

To train a tree in a random forest, at each split m variables are randomly selected from the global variable set, and the variable that maximizes information gain is chosen to be split on. Source: https://www.stat.berkeley.edu/~breiman/randomforest2001.pdf (the original random forest paper).

$\endgroup$
2
  • $\begingroup$ But once a particular set of variables is selected from the global set for a particular tree, the pool of available variables for that tree remains fixed right? By 'each split' I believe you mean at each tree. $\endgroup$ – Dhiraj Apr 8 '17 at 23:51
  • 2
    $\begingroup$ No, the pool of available variables is always drawn anew from the global variable set at each split. $\endgroup$ – liangjy Apr 9 '17 at 0:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.