3
$\begingroup$

I'm using sklearn Decision Tree Classifier with some continuous features. When I run export_graphviz I see the same features in more than one nodes and with different values. Example: enter image description here

I would like to take the top important ones and want to use feature_importances_ for that. The problem is that feature_importances_ is array without reference to the tree nodes. I have the original features but as each one can be more than one time in the tree I'm not sure how to relate importance to node.

$\endgroup$
3
$\begingroup$

I think you are mixing two different things here.

  1. The feature_importance_ - this is an array which reflects how much each of the model's original features contributes to overall classification quality.
  2. The features positions in the tree - this is a mere representation of the decision rules made in each step in the tree. A feature position(s) in the tree in terms of importance is not so trivial.

There are some potential heuristics for understanding the relation between the two. If a feature dosn't appear in the tree it has 0 importance and generally the higher the feature is in the tree the more important it is (assuming its being compared to another feature on the same branch).

| improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.