2
$\begingroup$

[EDIT] The question now has been solved, I updated the calculations bellow.

I've been trying to understand the math behind shap values. So far I understand all the concepts in SHAP but not to get to the shap values that are in this example (coming from the last example of https://shap.readthedocs.io/en/latest/example_notebooks/tabular_examples/tree_based_models/Understanding%20Tree%20SHAP%20for%20Simple%20Models.html ).

What I've got so far is : Assume you have to explain an observation where x0=0,x1=0,x2=0,x3=0. So x is a vector of zeros for this case. Since we know x2 and x3 are NOT in the tree we wont do any calculation with them (if it is a GBM tree based model, simply parse if the variable is in any of the trees of the ensemble). To proove this is correct just calculate shap without this values or with a vector of 100 zeros, shap importances greater than 0 won't change (x0 and x1 and local accuracy property will stil hold (local accuracy means "sum of contributions + expected value = predicted output ).

Calculate all possible expected values in the tree (remember that the predicted values with a set of features for the tree is the expected value conditioned on the features).

STEP 1 EXPECTED VALUES: Expected values (not conditioned and conditioned)

E(y) = 0.75

E(y|x0=0) = 0

E(y|x0=0,x1=0) = 0

E(y|x1=0)=0 * 0.5+ 0.5 * 1.0=0.5

STEP 2: Now calculate contributions

contribution_adding_x0_to_null_model= E(y|x0=0) - E(y) = 0 - 0.75 = -0.75

contribution_having_x1_to_null_model = E(y|x1=0) - E(y) =0.5 - 0.75 = -0.25

contribution_adding_x0_to_x1= E(y|x0=0,x1=0) - E(y|x1=0) = 0 - 0.5 = -0.5

contribution_adding_x1_to_x0 = E(y|x0=0,x1=0) - E(y|x0=0) = 0 - 0 = 0

This would be the average of all possible combinations (superset) of having this feature and not having that?

** UPDATE 2 **, correct calculation

shap_x0 = mean (contribution_adding_x0_to_null_model , contribution_adding_x0_to_x1 ) = mean(-0.75,-0.5) = (-0.75-0.5)/2=-0.675

shap_x1 = mean( contribution_adding_x1_to_null_model, contribution_adding_x1_to_x0 ) = mean(-0.25,0)=-0.125

Proof this is correct, see shap package output bellow and also (local accuracy):

Prediction when all x are zeros (see the tree leaf in the left)

prediction = E(y|x0=0,x1=0,x2=0,x3=0) = 0

shap_x1+shap_x0+expected_value = -0.125-0.675+0.75=0=prediction

What am I forgetting about? [DONE]

Code for generating the tree:

# build data
N = 100
M = 4
X = np.zeros((N,M))
X.shape
y = np.zeros(N)
X[:N//2, 0] = 1
X[:1 * N//4, 1] = 1
X[N//2:3 * N//4, 1] = 1
y[:1 * N//4] = 1
y[:N//2] += 1

# fit model
and_fb_model = sklearn.tree.DecisionTreeRegressor(max_depth=2)
and_fb_model.fit(X, y)

# draw model
dot_data = sklearn.tree.export_graphviz(and_fb_model, out_file=None, filled=True, rounded=True, special_characters=True)
graph = graphviz.Source(dot_data)
graph

enter image description here

Explain the model for all zeros or all ones:

xs = [np.ones(M), np.zeros(M)]
for x in xs:
    print()
    print("          x =", x)
    print("shap_values =", shap.TreeExplainer(and_fb_model).shap_values(x))
>>> Output of this code is:

>>>          x = [1. 1. 1. 1.]
>>> shap_values = [0.875 0.375 0.    0.   ]

--- THIS ONE IS THE CASE OF THE EXAMPLE I GAVE, x is a vector of zeros: x=(x0,x1,x2,x3) =(0,0,0,0)
>>>          x = [0. 0. 0. 0.]
>>> shap_values = [-0.625 -0.125  0.     0.   ]

Thanks in advance!

UPDATE 3: Additional if anyone is interested: SOLUTION WHEN vector x = (1,1,1,1)

STEP1 EXPECTATIONS

E(y) = 0.75

E(y|x0=1) = 1.5

E(y|x0=1,x1=1) = 2

E(y|x1=1)= 0.5*0 + 0.5+2 = 1.0

STEP2 Contributions

contribution_adding_x0_to_null_model = E(y|x0=1) - E(y) = 1.5 - 0.75 = 0.75

contribution_having_x1_to_null_model = E(y|x1=1) - E(y) = 1.0 - 0.75 = 0.25

contribution_adding_x0_to_x1 = E(y|x0=1,x1=1) - E(y|x1=1) = 2 - 1.0 = 1.0

contribution_adding_x1_to_x0 = E(y|x0=1,x1=1) - E(y|x0=1) = 2-1.5=0.5

shap_x0 = mean(contribution_adding_x0_to_null_model,contribution_adding_x0_to_x1) = mean(0.75,1.0)=0.875

shap_x1 = mean(contribution_having_x1_to_null_model,contribution_adding_x1_to_x0) = mean(0.25,0.5)=0.375

STEP 4: Check local accuracy property:

When x=1 , the prediction is:

prediction = 2.0

The sum of shap values is:

shap_x0 +shap_x1 + expected_value = 0.875+0.375+0.75=2.0

It definitely hodls.

$\endgroup$

1 Answer 1

4
+100
$\begingroup$

You seem to have entirely the right idea, you just miscalculated the second and fourth contributions you listed. Below are the corrected calculations, bolded to indicate where a change has been made:

contribution_having_x0 = E(y|x0=0) - E(y) = 0 - 0.75 = -0.75
contribution_having_x1 = E(y|x1=0) - E(y) = 0.5 - 0.75 = -0.25
contribution_adding_x0_to_x1 = E(y|x0=0,x1=0) - E(y|x1=0) = 0 - 0.5 = -0.5
contribution_adding_x1_to_x0 = E(y|x0=0,x1=0) - E(y|x0=0) = 0 - 0 = 0

After this, averaging the two contributions when adding a feature gives the same shap values as reported by the package. (The mistake in the formula for the second equation gives rise to your mistaken simplification of it.)

$\endgroup$
3
  • 2
    $\begingroup$ Thanks Ben! I guess I was burntout (let's say that , to justifiy my newbie mistakes). Mistake 1 was a typo, mistake 2 was I was adding the incorrect contributions. I updated the question with the calculatio when x=1 too, any suggestion is more than welcome. $\endgroup$
    – Tom
    Sep 30, 2022 at 21:13
  • 2
    $\begingroup$ @Tom np. I'm not sure what the best course of action is for this Q&A: as is, it's unlikely very helpful to future readers, and with your edits to the Q there isn't actually a question anymore. I can revoke the bounty (as a mod), and maybe the Q should just be deleted. But I think that'd be a pity, because tracking through the computations alongside the more-intuitive explanation in the shap example is useful. Hm. $\endgroup$
    – Ben Reiniger
    Sep 30, 2022 at 21:26
  • 1
    $\begingroup$ I went through the question and find it very interesting the thread. I will suggest to keep the Q $\endgroup$ Oct 1, 2022 at 19:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.