13
$\begingroup$

What is the parameter max_features in DecisionTreeClassifier responsible for?

I thought it defines the number of features the tree uses to generate its nodes. But in spite of the different values of this parameter (n = 1 and 2), my tree employs both features that I have. What changes so?

max_features = 2

enter image description here

max_features = 1

enter image description here

You can see x1 and x2 are used in both cases

Improve this question
$\endgroup$
5
  • $\begingroup$ In the documentation it is stated: "If int, then consider max_features features at each split". Thus, it it is the maximum number of features used in the condition at each node of the tree. Your example is misleading, because even in the case of max_features=2 your splits are using only one feature in the decisions. $\endgroup$
    – mapto
    Commented Nov 19, 2018 at 16:07
  • $\begingroup$ Could you provide any example as an answer, please? Because I could find graphs with only one feature condition at each node. For example, check first ten graphs from google.com/… $\endgroup$
    – James Flash
    Commented Nov 19, 2018 at 16:37
  • $\begingroup$ Possible duplicate of Is max_depth in scikit the equivalent of pruning in decision trees? $\endgroup$
    – MzdR
    Commented Nov 19, 2018 at 21:02
  • $\begingroup$ Hi James, it seems that my comment was also inaccurate. @Bashar Haddad's answer sounds more convincing to me. If it convinces you as well, I'd suggest you accept it. $\endgroup$
    – mapto
    Commented Nov 20, 2018 at 8:44
  • $\begingroup$ @MzdR, the two questions are about different parameters, thus not duplicate. $\endgroup$
    – mapto
    Commented Nov 20, 2018 at 8:44

1 Answer 1

Reset to default
17
$\begingroup$

Max_feature is the number of features to consider each time to make the split decision. Let us say the dimension of your data is 50 and the max_feature is 10, each time you need to find the split, you randomly select 10 features and use them to decide which one of the 10 is the best feature to use. When you go to the next node you will select randomly another 10 and so on.

This mechanism is used to control overfitting. In fact, it is similar to the technique used in random forest, except in random forest we start with sampling also from the data and we generate multiple trees.

So even if you set the number to 10, if you go deep you will end up using all the features, but each time you limit the set to 10.

If you compare the definition of the max feature in the decision tree and random forest, you will see that they are the same.

https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html

https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.