Skip to main content
7 votes

difference between feature effect and feature importance

In A Unified Approach to Interpreting Model Predictions the authors define SHAP values "as a unified measure of feature importance". That is, SHAP values are one of many approaches to ...
Jonathan's user avatar
  • 5,430
5 votes
Accepted

SHAP value analysis gives different feature importance on train and test set

Since SHAP gives you an estimation of an individual sample (they are local explainers), your explanations are local(for a certain instance) You are just comparing two different instances and getting ...
Carlos Mougan's user avatar
4 votes
Accepted

Can you do the math for this simple treeSHAP example (decisionTree)?

You seem to have entirely the right idea, you just miscalculated the second and fourth contributions you listed. Below are the corrected calculations, bolded to indicate where a change has been made: ...
Ben Reiniger's user avatar
  • 11.9k
4 votes
Accepted

Shapley values without intercept (or without `expected_value`)

This is very similar to fitting a linear regression and not including an intercept, and I think they will face similar issues. To be very concrete, consider an example with $f(x)=1,\ E=1, \ \phi_1=1, \...
Ben Reiniger's user avatar
  • 11.9k
4 votes
Accepted

Is it valid to compare SHAP values across models?

Shapley values were designed in the context of game theory (source), to share value created by a coalition of player in a game. It has multiple properties, including linearity. The linearity ensure ...
Lucas Morin's user avatar
  • 2,289
3 votes

How can SHAP feature importance be greater than 1 for a binary classification problem?

First, SHAP values are not directed translated as probabilities, they are marginal contributions for model's output. As explained in this post, we can't interpret SHAP values from raw predictions. ...
Victor Oliveira's user avatar
3 votes
Accepted

How is the "base value" of SHAP values calculated?

As you say, it's the value of a feature-less model, which generally is the average of the outcome variable in the training set (often in log-odds, if classification). With ...
Ben Reiniger's user avatar
  • 11.9k
3 votes

Is multicollinarity a problem when interpreting SHAP values from an XGBoost model?

Shapley values are designed to deal with this problem. You might want to have a look at the literature. They are based on the idea of a collaborative game, and the goal is to compute each player's ...
Carlos Mougan's user avatar
3 votes
Accepted

Why do Shapley value solutions remain consistent when the value function of the empty set changes in the ML context?

At a high level, this mostly goes away by just defining the valuation function $v$ as the expected model output given the coalition (*abusing terminology and notation a little) minus the global ...
Ben Reiniger's user avatar
  • 11.9k
2 votes

difference between feature effect and feature importance

SHAP values estimate the impact of a feature on predictions whereas feature importances estimate the impact of a feature on model fit.
bradS's user avatar
  • 1,615
2 votes

How can interparet shap.summary_plot and its gray color concerning outliers/anomaly?

I had the same question, and according to this link: Grey represents the categorical values which cannot be scaled in high or low. Concerning the other questions, I found this link: https://github....
Mariane Reis's user avatar
2 votes
Accepted

How to interpret SHAP summary plot?

How do I know which feature leads to class 1 and class 0? The length of the bar tells you how much influence the feature has on the prediction. Does it mean high values of each feature leads to class ...
Akavall's user avatar
  • 934
2 votes

Shap summary Plot for binary classification and multiclass

In binary classification, the shap values for the two classes, given a feature and observation, are just opposites of each other, so you get no added information by providing both. You can see this, ...
Ben Reiniger's user avatar
  • 11.9k
2 votes

Determining SHAP values for combined features

I'll try to answer in parts: On SHAP values being linear: SHAP values are calculated based on the marginal contribution of a given feature to the prediction, considering all possible combinations of ...
ABizz's user avatar
  • 141
1 vote
Accepted

shapley graph plot via python

You could have a look at the SHAP package. Here is a link to some similar examples. As described in these examples, the following code will produce the both plots: ...
Broele's user avatar
  • 1,525
1 vote
Accepted

Why does the SHAP library in python seem to be so poorly coded/maintained, and how do I get a working environment?

It seems like you want to vent. The first part of your title is not necessary (and the answer is because it is free). Would you rather have nothing at all ? Regarding your question, it is a bit too ...
Lucas Morin's user avatar
  • 2,289
1 vote

Can using the mean of absolute Shapely Values for feature importance give very wrong results?

I think you are correct in your analysis, but have used a very unusual context. It would be an impressively bad model that uses $A$ confidently in the wrong direction. If such a situation did occur, ...
Ben Reiniger's user avatar
  • 11.9k
1 vote

Why the marginal contribution of a feature is the difference between the feature effect minus the average effect

Here is a crucial paper to be read that I was also unaware of.$^\dagger$ The article points out the essential differences, both from a theoretical and empirical point of view, between feature ...
Eduard's user avatar
  • 669
1 vote

Shapley Values - How to interpret each value for each feature for a specific instance?

Depending on the explainer and whether feature_dependence="independent", you may be able to get the values in probability space directly with the option <...
oW_'s user avatar
  • 6,367
1 vote

Shap summary Plot for binary classification and multiclass

The importance is drawn in one color, because we have 1 class, why should we draw one value in several colors? For a multiclass task, shap is considered for each class, so the colors are different. ...
Andrew's user avatar
  • 406
1 vote

What is the difference (interpretation) between the partial R^2 and the SHAP value for a linear regression model?

I think one main difference is that SHAP value calculate the marginal contribution of the features across all the coalition, which includes the interaction between this features and other features. ...
RankieJiang's user avatar
1 vote

What is the difference (interpretation) between the partial R^2 and the SHAP value for a linear regression model?

Since no answers have been posted yet I will do my best.... SHAP values don't reflect the output's variation when a variable is removed from the model, as R2 does. SHAP values are the contribution of ...
skan's user avatar
  • 185
1 vote

Are predictive features with 0 SHAP values included in the model?

I am in the same situation, the analyzes carried out by me so far are: Missing data can influence: Studying my data I did not find a direct relationship, since I found the same compartment, for ...
user149297's user avatar
1 vote

What is the SHAP values for a liner model? How do we derive that?

The equation you showed is true only if the features are independent as well. Here is from the help of shap.LinearExplainer: ...
user135257's user avatar
1 vote

Shap after scikit pipeline - feature name

probably a bit late, but still. In sklearn, Pipeline/ColumnTransformer (and other) have usually function get_feature_names_out() returning feature names after ...
b4sus's user avatar
  • 11
1 vote

Shapley contribution when coalition is 0

In the blog post, the author defines the characteristic function $v$ to be the sum of conversion across all subsets of the coalition. (If I'm understanding the raw data correctly, this is the number ...
Ben Reiniger's user avatar
  • 11.9k
1 vote

How to achieve SHAP values for a CatBoost model in R?

catboost::catboost.get_feature_importance(model, pool = pool, type = "ShapValues")
JensMB's user avatar
  • 11
1 vote

SHAP value analysis gives different feature importance on train and test set

You have to make sure that the problem doesn't come from your data or your model : Make sure that your data don't change significantly (same % of classes) but also general distribution / correlation ...
Lucas Morin's user avatar
  • 2,289
1 vote
Accepted

Explanation of how DeepExplainer works to obtain SHAP values in simple terms

From https://en.wikipedia.org/wiki/Shapley_value, it is possible to understand that direct computation of Shapley values is difficult with their general formula : $$ \varphi_i(v) = \frac{1}{\text{...
Lucas Morin's user avatar
  • 2,289
1 vote
Accepted

Getting the positive impacting features using SHAP

Depending on your model there may be some better model-specific approaches than SHAP. It is also important to note that SHAP is an approximation of Shapley value, with the main assumption of not ...
Lucas Morin's user avatar
  • 2,289

Only top scored, non community-wiki answers of a minimum length are eligible