i am using XGboost to predict a 2 classes target variable on insurance claims. I have a model ( training with cross validation, hyper parameters tuning etc...) i run on another dataset.

My question is :

is there a way to know why a given claim has been affected to one class i.e. the features that explain the choice made by the model?

The purpose is to be able to justify the choice made by the machine to third party human.

Thanks for your answer.


3 Answers 3


I suggest you to go for Shap. It uses the Shapley values (concept borrowed from the Game Theory) to describe the model behavior, and with that it can explain a single prediction.

It's graphical interface uses Force Plots, like the one that you see below. enter image description here

The red bar is built by the features that lead the prediction to positive values, and the blue by the other ones.

In your case (a classifier) the number in bold will be the one just before the sigmoid function that will limit the output value between zero and one (one class or the other). So don't be scared if in some cases it will be greater than one, or negative.

The size of the segments represent how much that feature contribute to the prediction, and under the segments you see the name of the feature (ex. LSTAT) and it's actual value (ex. 4.98). So, in this case, LSTAT is the mean feature that lead the prediction for that element of the dataset to the value of 24.41 (the number in bold).



You can use the ELI5 library to explain the feature contributions to individual predictions for XGBoost models.

See Explaining Predictions in the docs, copied below:

To get a better idea of how our classifier works, let’s examine individual predictions with eli5.show_prediction():

from eli5 import show_prediction
show_prediction(clf, valid_xs[1], vec=vec, show_feature_values=True)

enter image description here


thank you for the answers.

This R package seems to do the job :



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.