Feature Importance

Question

I have a dataset with 10 features.

1. I've computed the feature importance using permutation importance with cross-validation from eli5, after fitting an extremely randomized trees (ET) classifier form Scikit learn.

2. I fitted 10 different ETs using only one feature at a time and computed the mean cross-validation score using the same CV scheme.

I noticed while the order of importance of the features that I get from the permutation importance matches the order of features when I rank them using the mean CV scores I get when fitting model with only one feature at a time, there is one feature that permutation importance classes at the very bottom while its mean CV scores when the model is fitted using it alone is the second one.

Suppose that from 1. the order of the features is feature2, feature4, feature5, feature1, feature9, feature8, feature7, feature10, feature3, feature6. When using one feature at a time and computing the mean cv score, I noticed that the mean cv score of feature 3 is the second-best, although the importance feature is suggesting it is a weak feature.

I would appreciate if anyone can help me understand how to interpret this and which approach seems to be more trustworthy.

Thank you.

Answer 1

Lets first look at how the algorithm for permutation importance works. As per the documentation:

To avoid re-training the estimator we can remove a feature only from the test part of the dataset, and compute score without using this feature. It doesn’t work as-is, because estimators expect feature to be present. So instead of removing a feature we can replace it with random noise - feature column is still there, but it no longer contains useful information. This method works if noise is drawn from the same distribution as original feature values (as otherwise estimator may fail). The simplest way to get such noise is to shuffle values for a feature, i.e. use other examples’ feature values - this is how permutation importance is computed.

Now, the answer to your question is that all though the feature 3 provide important information (cv score second best in scenario 2), that information can also be captured using the rest 9 features combined (so it is useless in scenario 1). While using multiple features, a feature is important if the model gains any new insights from it which the rest of the features cannot provide.

You can interpret this from the "Algorithm" part of ELI5 Permutation Importance.