Let's say I have three models:

  • a random forest with 100 trees
  • a random forest with 1000 trees
  • an xgboost model.

I can rank the importance of my features on my dataset for each model using SHAP, and compare relative importance across models. What is not clear is whether I can meaningfully compare the actual numerical SHAP values across models.

I am using the same features for all models.

  • $\begingroup$ "the archaic points system prevents me from commenting" - you can always comment in your own posts $\endgroup$
    – desertnaut
    Oct 17 '20 at 13:44

Shapley values were designed in the context of game theory (source), to share value created by a coalition of player in a game. It has multiple properties, including linearity. The linearity ensure that if you were to average your models, the resulting Shapley value would be the average of Shapley values for individual models. Shapley values are comparable in that sense of considering an average model.

I think the general answer would be the opposite. Inutitively, because looking at an individual Shapley value you can't know if the value is due to an 'individual performance' or 'overall performance' of the coalition. So looking at two values in the context of ML, the difference might be both explained by a different contribution of the individual value but also by a difference in the overall performance of the model. So I would avoid doing that in the general case. (But I often do so - along with comparing individual predictions - to check if two models with similar overall performance have learned the same things or not)

Overall I would suggest you use a more suited criteria (like an information criteria) for your model selection, then use Shapley value to explain the model you selected. Not using Shapley Values to do some sort of model selection.

Note that I am mainly talking about Shapley values and not SHAP, which is an approximation. You need to be cautious with SHAP as the approximation rely on the absence of correlation between your features, which rarely happen in practice.

  • $\begingroup$ It sounds like your answer implies that you could potentially use differences in Shapely values across models to explain why those models perform differently? I do understand that it's not a guarantee that for a given feature the "better" model produces a "more ideal" Shapely value of course because the performance is of the model as a system. $\endgroup$
    – DKL
    Dec 10 '21 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.