XGBoost model has features whose feature importance equal zero

Question

I ran into this problem:

A XGBoost model(.pickle file , constrcuted under V0.7.post3) with 100 features in it ; But I found 55 features in model (model.feature_importances_) show 0 feature importance (like matrix below);

Additionally, when I transformed the pickle file to PMML(to launch online), only 45 features in PMML file (those ones with importance>0 apparently);

So, my question is:

--why features with importance equal to 0 ending up in a XGB model ? And why they remain in the model, if they don't actually contribute to/participate in split?

## Feature importance maxtrix from model for demo
array([0.06586827, 0.04191617, 0.08383234, 0.05988024, 0.07784431,
   0.04790419, 
   ...
 
   0.        , 0.        , 0.        , 0.        , 0.        ,
   0.        , 0.        , 0.        , 0.        , 0.        ]

Answer 1

XGBoost Feature importance gives feature importance for all the features irrespective of they contribute to the model or not. So if your train data has 100 variable xgboost feature importance will give 100 feature importances though variables which are not important will be zero

Answer 2

xgboost simply displays the importance of the functions of the dataset on which it was trained, no more. If some functions are so bad that they are not included in any of the trees, then their importance will be 0. You can submit a dataset of size (a x b), and specify n_estimators=1,max_depth=1 and you will see that 1 function will have an importance of 1, all the others 0, because that they didn't even have a chance to get into the trees.

You can use some feature selection algorithms to get rid of unnecessary functions, such as boruta, recursive feature elimination, etc.