3
$\begingroup$

I am building a XGBoost model with Python and trying to explain it using the beautiful shap package. Apart from calculating SHAP values of each feature, I'd like to show graphs such as the two that follow (respectively a summary plot Figure A and a dependence plot Figure B):

importance plot dependence plot

This is easy if I have a single model, with a single train/test split. However, this is risky as I might have a selection bias, so I usually evaluate the accuracy of my models with 500 random train/test splits (then calculate average values of Accuracy and Cohen's Kappa).

If I do so, how can I get the above plots, considering average information from all the 500 XGBoost models? Three strategies come to my mind, but none seems to solve the issue:

1- Train a model on all samples (without split) and calculate SHAP values on that. I would keep calculating accuracy and Kappa on the 500 models with train/test split.

2- Select the model of 500 with the best fit on the test set and calculate SHAP values on that.

3- I can calculate SHAP values for each feature, for each of 500 repetitions, and then calculate their mean and standard deviation. Then I can produce a histogram out of that.

N.1 does not seem to be the correct approach. N.2 also seems to be biased, and things might change from one model to the other. N.3 is what I have been doing so far, as it seems to be the most reliable approach. However, it has the drawback that I cannot produce the graphs that I would like to have (Figure A and B).

Any idea to produce meaningful graphs (A and B) considering information from a repeated train/test split approach?

$\endgroup$
1
  • 1
    $\begingroup$ 500 random splits seems way too big. Maybe if you did less splits, you would be able to perform SHAP on each of those split. $\endgroup$ Jan 24, 2020 at 16:06

3 Answers 3

1
$\begingroup$

I think this depends in part on why you want the shap values. It can be helpful to separate (a) explaining the model, and (b) explaining the data; model explainability tools like shap only address (a), which hopefully serves as a proxy for (b).

If all you want is model explanations [(a)], then I think your approach 1 is fine; that retrained model is the one going into production anyway, presumably.

If you want data explanations [(b)], then part of the question you must ask is "how good a proxy" is (a) for (b), and one aspect of that is "how stable are the shap values?". In that case, you really want something like your approach 3; maybe not at the level of individual predictions like your two plots, but globally how importance is attributed to each feature, and how variable those importances are. Of course, if you go this route, it would be easy enough to follow @bmwilly's answer to generate your plots; you lose information about the variability of the estimates, but you get back to your nice plots.

Approach 2 seems like a poor one; you'll be overfitting to your test set, and biasing the shap values because of it.

$\endgroup$
2
$\begingroup$

I think approach number 3 is the correct one and you can indeed make plots doing it in this way.

This example does so using cross-validation. It's maybe a little overkill but you can adapt the code to your situation. The article also uses mean SHAP values from each fold.

$\endgroup$
1
$\begingroup$

I'd propose doing a similar thing that you do with your accuracy and Kappa metrics — calculate SHAP values for all 500 splits, and take the average of these $n\_samples \times n\_features \times 500$ matrixes in the third dimension to get $n\_samples \times n\_features$ matrixes, which you can use to create your desired plots.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.