# Decision Trees Should We Discard Low Importance Features?

I just started to work with feature selection. Let's say I have a decision tree model. I get its feature importances by tree.feature_importances_.

In my model out of around 30 features, 20 of them has importance value of zero. Does that mean that I should discard those low importance value features from my model? As I understood the answer is no, but I don't know the reason behind it. Can anyone explain?

• An importance value zero (at least for Gini importance, used by sklearn) indicates that the tree never splits on the feature. So removing it won't change the model. As to other low-importance features, I defer to the answer(s). Aug 17, 2019 at 21:53

As for many questions, the answer is "it depends":

• features which have a low individual importance may still add predictive power to your model, because the model benefits from combining their information together with information of other features.
• However they may introduce noise in the model and cause overfitting, thus decreasing the performance of the model.