0
$\begingroup$

I am very confused about how decision trees select features and threshold within each feature to do the split. I totally understand the different splitting metrics (Gini index and so on) used and how they work. But my problem is how sklearn chooses features and thresholds to calculate these metrics.

The estimator sklearn.tree.DecisionTreeClassifier has a parameter splitter. Let me admit that all the resources available online are not that good in explaining this parameter and they are conflicting each other. I still don't understand what will happen if I set splitter="best": does this means that the algorithm will consider all the features with all of its values to get the best threshold value? And in this case max_features attribute will not have any effect? And if I set splitter="random" the algorithm will randomly select certain numbers of features = max_features from the features and search each for certain random values of each of these features to find the threshold to split?

$\endgroup$

1 Answer 1

1
$\begingroup$

if I set splitter="best", does this means that the algorithm will consider all the features with all of its values to get the best threshold value ?? and in this case max_features attribute will not have any effect ?

Not quite: max_features still has an effect here. max_features features are selected at random, but for each of those, the best among all possible thresholds is selected.

And if I set splitter="random" the algorithm will randomly select certain numbers of features = max_features form the whole features and search each for certain random values of each of these features to find the threshold to split ?

Right, max_features has the same effect regardless of the splitter, but when splitter="random", instead of testing every possible threshold for the split on a feature, a single random threshold, drawn uniformly between the feature's minimum and maximum, is tested. Source code

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.