I am building a XGBoost model with Python and trying to explain it using the beautiful shap package. Apart from calculating SHAP values of each feature, I'd like to show graphs such as the two that follow (respectively a summary plot Figure A and a dependece plot Figure B):
This is easy if I have a single model, with a single train/test split. However, this is risky as I might have a selection bias, so I usually evaluate the accuracy of my models with 500 random train/test splits (then calculate average values of Accuracy and Cohen's Kappa).
If I do so, how can I get the above plots, considering average information from all the 500 XGBoost models? Three strategies come to my mind, but none seems to solve the issue:
1- Train a model on all sample (without split) and calculate SHAP values on that. I would keep calculating accuracy and Kappa on the 500 models with train/test split.
2- Select the model of 500 with the best fit on test set and calculate SHAP values on that.
3- I can calculate SHAP values for each feature, for each of 500 repetitions and then calculate their mean and standard deviation. Then I can produce an histogram out of that.
N.1 does not seem to be the correct approach. N.2 also seems to be biased, and things might change from one model to the other. N.3 is what I have been doing so far, as it seems to be the most reliable approach. However, it has the drawback that I cannot produce the graphs that I would like to have (Figure A and B).
Any idea to produce meaningful graphs (A and B) considering information from a repeated train/test split approach?