0
$\begingroup$

When I look at decision trees, they start with a root node choosing the most suitable feature from which to split further. What if the decision tree is unable to find the most suitable feature from the data as root node? How does the decision tree cope in that situation?

$\endgroup$
2
  • $\begingroup$ Why would they be unable to find a suitable feature? Do you mean if the metric for two features is a tie or something? $\endgroup$
    – Dan Scally
    Aug 21 '19 at 7:48
  • $\begingroup$ yes, something like that. Not all data-sets have an obvious winning feature. $\endgroup$
    – user781486
    Aug 21 '19 at 7:48
1
$\begingroup$

In scikitlearn at least the code is in this file: https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/tree/_splitter.pyx.

Basically the determination of the root node is the same as the determination of any other node, it's just the first one that's done. What I think it's doing is either random splits (in which case your question is irrelevant) or "best" split, where best is determined by the evaluation of some criterion like decrease of gini impurity for each feature. In the case of a "Best" split it's evaluating the criterion for each feature in turn and keeping track of the best one, only overwriting that where a new feature's criterion evaluation is strictly better than the currently stored best feature. That strictly better evaluation means that, where two features have the same value for the criterion, the model will select the first feature it encounters as the one to split.

In other words, if Feature A and Feature B would result in an equally good split, it will set the root node to be Feature A purely because it evaluated that first.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.