I have been using DeepExplainer (DE) to obtain the approximate SHAP values for my MLP model. I am following the SHAP Python library.

Now I'd like learn the logic behind DE more. From the relevant paper it is not clear to me how SHAP values are gotten. I see that a background sample set is given and an expected model output is calculated based on this data and the difference is calculated with the current model's output. This difference is the sum of the SHAP values. However, I don't understand how each contribution is obtained? Could you give an explanation with simple terms?

  • 1
    $\begingroup$ Deeplift and Shapley values run in the backend which get derived pretty much like a coalitional game. Have you gone through this (en.wikipedia.org/wiki/Shapley_value)? $\endgroup$ Aug 30 '19 at 12:48

From https://en.wikipedia.org/wiki/Shapley_value, it is possible to understand that direct computation of Shapley values is difficult with their general formula :

$$ \varphi_i(v) = \frac{1}{\text{number of players}} \sum_{\text{coalitions excluding }i} \frac{\text{marginal contribution of }i\text{ to coalition}}{\text{number of coalitions excluding } i \text{ of this size}}$$

Basically beacuse the number of coalitions excluding i grows in complexity with $\sum_{k=1}^{n-1} k!$, where n is the number of variables. Some progresses has been made in the direction of evaluating this sum with Monte-Carlo techniques (as mentionned in https://christophm.github.io/interpretable-ml-book/), but those calculations are still intensive.

In their article (http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions) Lundberg and Lee proposes two new approaches, relying on SHAP - (these are the Shapley values of a conditional expectation function of the original model) :

  • A model agnostic approach basically rewriting the problem as a linear regression problem, which is intuitively less expensive to compute. Basically saying that SHAP is a function of the weight of the model and trying to approximate it.
  • A model specific approach. Assuming input independence (which is rarely true ...) they show how to compute SHAP value directly from model weights. Starting with linear models they devise similar relations for NN with usual propagation techniques.

As to which method is used exactly for MLP, I don't know exactly, but the second one seems more appropriate (model specific, exact method).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.