Feature Importance From Xgboost:
('A', 20.263429)
('B', 14.631438)
('C', 49.617475)
('D', 1.7183341)
('E', 0.0)
('F', 4.438192)
('G', 4.4471968)
('H', 0.75913663)
('I', 4.1248)

Feature Importance From Catboost:
('A', 31.77073440713819)
('B', 25.450736642294263)
('C', 4.584613760816324)
('D', 1.2102069060769505)
('E', 0.2570517403500689)
('F', 4.556130077381411)
('G', 7.5309680136560075)
('H', 2.0941628837404185)
('I', 22.54539556854636)

As you can see the feature importance of 'C' is way too low in Cat boost.


There is no guarantee that you get the same subset of features from any feature selection algorithm. There might be multiple subsets that all perform approximately equally. For instance, perhaps feature C is highly correlated with feature I. So, there is no reason to be surprised by what you are observing. There is no notion of "too low".


The subsets of features that you select might vary between algorithms and also between parameters of the same algorithm.

You can try, for the same algorithm try different hyperparameters, the feature importance might change.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.