Let say I got feature importance for xgclassifier

sorted(zip(xgb.feature_importances_, X.columns), reverse=True)

[(0.10650729, 'modelMag_i'),
 (0.08187373, 'psfMag_g'),
 (0.070714064, 'modelVar'),
 (0.06747197, 'modelMag_z'),
 (0.061302684, 'fiberMag_g'),
 (0.05923392, 'fibVar'),
 (0.057112347, 'psfMag_u'),
 (0.05275245, 'psfMag_r'),
 (0.047756154, 'modelMag_g'),
 (0.046770878, 'psfMag_z'),
 (0.034744404, 'modelMag_r'),
 (0.034687676, 'psfMag_i'),
 (0.032622278, 'petroMag_i'),
 (0.028391415, 'modelMag_u'),
 (0.025683628, 'petroMag_r'),
 (0.024703711, 'petroMag_z'),
 (0.022656566, 'fiberMag_z'),
 (0.021865964, 'petroMag_g'),
 (0.01854887, 'fiberMag_r'),
 (0.018389946, 'fiberMag_u'),
 (0.01721868, 'modelMean'),
 (0.016091293, 'fiberMag_i'),
 (0.013110901, 'fibMean'),
 (0.011618578, 'modelSum'),
 (0.010491995, 'fiberID'),
 (0.008898865, 'fibSum'),
 (0.008779789, 'petroMag_u')]

is removing the lowest feature will improve for xgboost or lgb classifier? or xgboost or lgb does not matter with feature importance


1 Answer 1


There is no certain answer, only trial and error. Though it should help.

Let me elaborate. Feature importance shows the impact of features on the quality of the model: the number of times there was a split using this feature or gains from splitting on this feature. The better is the feature, the higher is the importance. But some features could be important due to interactions with other features. Also if some features have a high correlation between them, the importance value could be split between them. In general removing low importance features should have a small impact on the metric. It could be positive or negative. Just try and see :)

  • 1
    $\begingroup$ Thank you. So there is no such guide line or prove 'yet' for this topic right? is there any good or recent research paper abut this topic? $\endgroup$
    – slowmonk
    Commented Feb 18, 2020 at 4:16
  • $\begingroup$ Yes, there is no clear rule in this case. I think this article could be useful: explained.ai/rf-importance $\endgroup$ Commented Feb 18, 2020 at 17:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.