Can we calculate the difference between Shapley values to interpret changes in the output? More precisely, if we get Shapley values for two different inputs, can we compare them to understand how much each feature led to the increase/decrease of the output?

Example: Let's say that I have a regression model with 10 features as input: $x=(x_{1}, x_{2}, x_{3}, ...,x_{10})$. I get two predictions: $y^{1}=70$ and $y^{2}=100$ and I want to understand how much each feature has led to such an increase (+30). Let's say that for the two inputs $x = (x_{1}, x_{2}, x_{3}, ..., x_{10})$ and $x' = ({x_{1}', x_{2}', x_{3}', ..., x_{10}'})$ I get the following Shapley values: $\phi = (7, -4, 6, 18, 7, 0, 14, -3, 17, 7)$ and $\phi' = (7, -6, 6, 20, 7, 0, 32, -3, 27, 9)$. Here I'm assuming that the expected value of all the prediction is 1 (this is why the Shapley values sum up to 69 and 99 respectively).

I calculate $\phi'-\phi = (\phi'_{1}-\phi_{1}, \phi'_{2}-\phi_{2}, ..., \phi'_{10}-\phi_{10})$ which in my example is going to be equal to: (0, -2, 0, 2, 0, 0, 18, 0, 10, 2).

  • Is it correct to say that the 2nd feature led to a decrease of -2, the 4th feature led to an increase of 2, the 7th feature to an increase of 18, etc.?

I believe this can be inferred somehow from the additive property of Shapley values, but I'm not finding the connection yet.

  • 1
    $\begingroup$ Cannot see anything wrong with your rationale - as you say, it seems to follow directly from the additive property of the Shapley values. $\endgroup$
    – desertnaut
    Oct 17, 2020 at 10:52


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.