# Get feature importance for each observation with XGBoost

I have trained an XGBoost binary classifier and I would like to extract features importance for each observation I give to the model (I already have global features importance).

More specifically, I am looking for a way to determine, for each instance given to the model, which features have the most impact and make the input belong to one class or another. I would like to know something like the top 5 features which make the observation belong to some class and indications on how I should modify these 5 features so that the probability of belonging to this class decreases or increases.

For example, let’s say my model predicts whether a house costs more than 100,000 dollars (this is the positive class) based on its location, surface and number of bedrooms. I give it the following input: London, 400 square foots, 4 bedrooms and my model predicts a probability of 56% for the house to be in the positive class. I am looking for a Python module or a function that would show the most influential features for each observation.

• did you get any solution on this ? Jan 7 '21 at 10:19

I would suggest you're probably looking for something like one of these two packages:

See this notebook of lime's for example, which shows how you can use it to see why a specific sample in your data resulted in the prediction from the model:

https://marcotcr.github.io/lime/tutorials/Tutorial%20-%20continuous%20and%20categorical%20features.html

After training your model, use xgb_feature_importances_ to see the impact the features had on the training. Note that there are 3 types of how importance is calculated for the features (weight is the default type) :

• weight : The number of times a feature is used to split the data across all trees.
• cover : The number of times a feature is used to split the data across all trees weighted by the number of training data points that go through those splits.
• gain : The average training loss reduction gained when using a feature for splitting.

Here's an example :

#Available importance_types = [‘weight’, ‘gain’, ‘cover’, ‘total_gain’, ‘total_cover’]
f = 'gain'
xgb.get_booster().get_score(importance_type= f)

• I'm not sure this answers OP's question, as they state they already have global feature importance. Aug 2 '19 at 13:35
• didn't see that. oops. Aug 2 '19 at 14:46