1
$\begingroup$

I'm making a using DecisionTreeClassifier from SKlearn (v0.21.3) with its default settings, using Python. I do not want regularize it in any way, I want it to overfit as much as possible.

When drawing the tree out I saw that some of the leafs were not pure. Is this normal? Was the tree not able to separate the samples? 

   ...
   model = DecisionTreeClassifier(criterion="entropy")
   model = modell.fit(X, y) 
   ...

enter image description here

Improve this question
$\endgroup$
2
  • $\begingroup$ The default settings include Gini impurity as the criterion, not entropy. Are you sure you set all other values to default / can you share the code sample? $\endgroup$
    – Romain Reboulleau
    Commented Nov 15, 2019 at 6:12
  • $\begingroup$ One reason for the tree not splitting some nodes could be that observations sharing the exact same features have different classes. Do you have such data? $\endgroup$
    – Romain Reboulleau
    Commented Nov 15, 2019 at 6:14

1 Answer 1

Reset to default
2
$\begingroup$

With default settings the DecisionTreeClassifier does not have any restrictions in terms of complexity as described in the Scikit Documentation.

Therefore, it will stop further branching the tree if either a given node is pure (all examples have the same classification) or there are no further attributes to branch on.

So if your final tree contains leaves which are not pure (for the data set it has been trained on) the algorithm did not have any attributes to further split on.

In case you applied any kind of randomization on your data like a random split of training and test data this might turn out differently when splitting again and getting a different training set.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.