feature importance and xgboost?

Question

Let say I got feature importance for xgclassifier

sorted(zip(xgb.feature_importances_, X.columns), reverse=True)

[(0.10650729, 'modelMag_i'),
 (0.08187373, 'psfMag_g'),
 (0.070714064, 'modelVar'),
 (0.06747197, 'modelMag_z'),
 (0.061302684, 'fiberMag_g'),
 (0.05923392, 'fibVar'),
 (0.057112347, 'psfMag_u'),
 (0.05275245, 'psfMag_r'),
 (0.047756154, 'modelMag_g'),
 (0.046770878, 'psfMag_z'),
 (0.034744404, 'modelMag_r'),
 (0.034687676, 'psfMag_i'),
 (0.032622278, 'petroMag_i'),
 (0.028391415, 'modelMag_u'),
 (0.025683628, 'petroMag_r'),
 (0.024703711, 'petroMag_z'),
 (0.022656566, 'fiberMag_z'),
 (0.021865964, 'petroMag_g'),
 (0.01854887, 'fiberMag_r'),
 (0.018389946, 'fiberMag_u'),
 (0.01721868, 'modelMean'),
 (0.016091293, 'fiberMag_i'),
 (0.013110901, 'fibMean'),
 (0.011618578, 'modelSum'),
 (0.010491995, 'fiberID'),
 (0.008898865, 'fibSum'),
 (0.008779789, 'petroMag_u')]

is removing the lowest feature will improve for xgboost or lgb classifier? or xgboost or lgb does not matter with feature importance

Answer 1

There is no certain answer, only trial and error. Though it should help.

Let me elaborate. Feature importance shows the impact of features on the quality of the model: the number of times there was a split using this feature or gains from splitting on this feature. The better is the feature, the higher is the importance. But some features could be important due to interactions with other features. Also if some features have a high correlation between them, the importance value could be split between them. In general removing low importance features should have a small impact on the metric. It could be positive or negative. Just try and see :)