i am using XGboost to predict a 2 classes target variable on insurance claims. I have a model ( training with cross validation, hyper parameters tuning etc...) i run on another dataset.
My question is :
is there a way to know why a given claim has been affected to one class i.e. the features that explain the choice made by the model?
The purpose is to be able to justify the choice made by the machine to third party human.
Thanks for your answer.
I suggest you to go for Shap. It uses the Shapley values (concept borrowed from the Game Theory) to describe the model behavior, and with that it can explain a single prediction.
It's graphical interface uses Force Plots, like the one that you see below. 
The red bar is built by the features that lead the prediction to positive values, and the blue by the other ones.
In your case (a classifier) the number in bold will be the one just before the sigmoid function that will limit the output value between zero and one (one class or the other). So don't be scared if in some cases it will be greater than one, or negative.
The size of the segments represent how much that feature contribute to the prediction, and under the segments you see the name of the feature (ex. LSTAT) and it's actual value (ex. 4.98). So, in this case, LSTAT is the mean feature that lead the prediction for that element of the dataset to the value of 24.41 (the number in bold).
Enjoy!
You can use the ELI5 library to explain the feature contributions to individual predictions for XGBoost models.
See Explaining Predictions in the docs, copied below:
To get a better idea of how our classifier works, let’s examine individual predictions with
eli5.show_prediction():from eli5 import show_prediction show_prediction(clf, valid_xs[1], vec=vec, show_feature_values=True)
thank you for the answers.
This R package seems to do the job :
https://medium.com/applied-data-science/new-r-package-the-xgboost-explainer-51dd7d1aa211
