Skip to main content

Questions tagged [shap]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
14 votes
2 answers
8k views

SHAP value analysis gives different feature importance on train and test set

Should SHAP value analysis be done on the train or test set? What does it mean if the feature importance based on mean |SHAP value| is different between the train and test set of my lightgbm model? ...
pbk's user avatar
  • 143
6 votes
1 answer
436 views

Shapley values without intercept (or without `expected_value`)

I have a model and I want to derive its interpretability by using feature contributions. In the end, I want to have some contribution per feature such that the sum of contributions equals the ...
David Masip's user avatar
  • 6,101
6 votes
1 answer
3k views

Is it valid to compare SHAP values across models?

Let's say I have three models: a random forest with 100 trees a random forest with 1000 trees an xgboost model. I can rank the importance of my features on my dataset for each model using SHAP, and ...
DKL's user avatar
  • 88
6 votes
1 answer
793 views

How to achieve SHAP values for a CatBoost model in R?

I'm asked to create a SHAP analysis in R but I cannot find it how to obtain it for a CatBoost model. I can get the SHAP values of an XGBoost model with ...
user100740's user avatar
5 votes
1 answer
9k views

How is the "base value" of SHAP values calculated?

I'm trying to understand how the base value is calculated. So I used an example from SHAP's github notebook, Census income classification with LightGBM. Right after I trained the lightgbm model, I ...
David293836's user avatar
5 votes
1 answer
2k views

Explanation of how DeepExplainer works to obtain SHAP values in simple terms

I have been using DeepExplainer (DE) to obtain the approximate SHAP values for my MLP model. I am following the SHAP Python library. Now I'd like learn the logic behind DE more. From the relevant ...
mlee_jordan's user avatar
4 votes
1 answer
227 views

Why do Shapley value solutions remain consistent when the value function of the empty set changes in the ML context?

Hey there data science stack exchange - question about SHAP. In the original Shapley value formulation from Lloyd, one assumption is that the value function of the empty set equals zero, $v(\emptyset) ...
shay's user avatar
  • 143
4 votes
1 answer
478 views

Is multicollinarity a problem when interpreting SHAP values from an XGBoost model?

I'm using an XGBoost model for multi-class classification and is looking at feature importance by using SHAP values. I'm curious if multicollinarity is a problem for the interpretation of the SHAP ...
hideonbush's user avatar
3 votes
2 answers
6k views

difference between feature effect and feature importance

Is there a difference between feature effect (eg SHAP effect) and feature importance in machine learning terminologies?
xlra's user avatar
  • 31
3 votes
0 answers
1k views

Differences between Feature Importance and SHAP variable importance graph

I have run an XGBClassifier using the following fields: ...
Giampaolo Levorato's user avatar
3 votes
0 answers
702 views

Explain FastText model using SHAP values

I have trained fastText model and some fully connected network build on its embeddings. I figured out how to use Lime on it: complete example can be found in Natural Language Processing Is Fun Part 3: ...
Mikhail_Sam's user avatar
2 votes
1 answer
319 views

Can you do the math for this simple treeSHAP example (decisionTree)?

[EDIT] The question now has been solved, I updated the calculations bellow. I've been trying to understand the math behind shap values. So far I understand all the concepts in SHAP but not to get to ...
Tom's user avatar
  • 65
2 votes
2 answers
653 views

What is the difference (interpretation) between the partial R^2 and the SHAP value for a linear regression model?

To calculate the coefficient of partial determination R2 for a given variable: We calculate the R2 with and without that variable and substract them. This implies fitting a different model with and ...
skan's user avatar
  • 185
2 votes
1 answer
4k views

How to interpret SHAP summary plot?

I already referred these posts here and here. So, please don't mark it as duplicate I am doing a binary classification using random forest and class labels are 1 and 0. What is the likelihood that ...
The Great's user avatar
  • 2,575
2 votes
1 answer
4k views

How can interparet shap.summary_plot and its gray color concerning outliers/anomaly?

I inspired by this notebook, and I'm experimenting IsolationForest algorithm using scikit-learn==0.22.2.post1 for anomaly ...
Mario's user avatar
  • 400
2 votes
1 answer
98 views

Shapley contribution when coalition is 0

I am exploring Shapley for channel attribution based on [here][1] Consider C1, C2, C3, C4 as 4 channels in question. Some of the coalition does not have value, such as ...
Kenny's user avatar
  • 121
2 votes
1 answer
1k views

Getting the positive impacting features using SHAP

I'm attempting to use SHAP to automatically extract feature names that have a positive impact on my regression models. On inspection of the code I see that the bar plot, for example, determines these ...
amateurjustin's user avatar
2 votes
1 answer
76 views

treeExplainer algorithm intuition

I'm reading the paper about the treeExplainer; the pseudo-code of Algorithm 1 is a bit cryptical as most of the variables are not even defined (same with sampling and all details involved). Is there a ...
zzzbob's user avatar
  • 45
2 votes
1 answer
1k views

How to calculate joint feature contribution for XGBoost Classifier in python?

I referred to this beautiful document to research about joint feature contibutions. But this works only for RandomForest algorithms because of treeinterpreter (does ...
Sudhakar Samak's user avatar
2 votes
0 answers
447 views

Capturing the 'direction' of feature importances using TreeSHAP?

I'm a machine learning / python novice, so apologies if my question is simple, but I haven't been able to find it addressed. I'm very interested in using ML to determine the most important features ...
agun7's user avatar
  • 21
2 votes
0 answers
943 views

Sample size for SHAP explainer and range of a SHAP value

I am working on a binary classification with 977 records with 77:23 class proportion. I used random forest model. Based on my attempt to run SHAP package, I got the below plots And I also see that ...
The Great's user avatar
  • 2,575
2 votes
0 answers
254 views

Shapley values for channel attribution equal to linear attribution

I am looking into Shapley values for online marketing attribution. In recent time many articles seem to have been made on this particular approach to attribution (there are more): https://medium.com/...
Tpeeters's user avatar
2 votes
0 answers
238 views

Shapley summary plot interpretation doubt?

I have question when interpreting SHAP summary plot. I have attached the sample plot Here, If I am interpreting it correctly, low values of feature 1 are associated with high and negative values for ...
Raghav Goyal's user avatar
2 votes
0 answers
790 views

How SHAP value explains contribution of features for outliers event?

I'm trying to understand and experiment with how the SHAP value can explain behaviour for each outlier events (rows) and how it can be related to shap.force_plot(). ...
Mario's user avatar
  • 400
2 votes
0 answers
466 views

Aggregate SHAP importances from different models

A couple of questions on the SHAP approach to the estimation of feature importance. I would like to use the random forest, logistic regression, SVM, and kNN to train four classification models on a ...
CaffeineMan's user avatar
1 vote
2 answers
3k views

Shap summary Plot for binary classification and multiclass

For binary classification, I am getting only a unicolor feature importance plot (i.e., the two classes do not appear individually). However, for multiclass, I am getting feature importance in ...
Muppi's user avatar
  • 11
1 vote
1 answer
999 views

How can SHAP feature importance be greater than 1 for a binary classification problem?

Let's say I build a binary classification model to predict survival on the Titanic. I then use SHAP to get feature importance for each feature. I see that the SHAP importance for the top feature, <...
Gaurav Bansal's user avatar
1 vote
1 answer
63 views

shapley graph plot via python

Does anybody know how can we plot this graph? Sth like attached.
Turan's user avatar
  • 113
1 vote
1 answer
248 views

Why the marginal contribution of a feature is the difference between the feature effect minus the average effect

In several sources the marginal contribution is defined as the difference between the prediction with and without the feature. However, recently I read an article where the marginal contribution was ...
Amir Jalilifard's user avatar
1 vote
1 answer
658 views

Are predictive features with 0 SHAP values included in the model?

I have trained and XGBoost by enforcing no-feaure interaction and calculated Global Shap values: It looks like only 6 features have some SHAP values, whilst the remaining ones have a SHAP value of 0. ...
Giampaolo Levorato's user avatar
1 vote
0 answers
21 views

Interpreting the SHAP values presented in layered violin plot in SHAP-library for Scikit binary RandomForestClassifier

I am using the SHAP-library for computing feature Shapley values for a binary RandomForestClassifier which has naturally two outputs, 0 or 1. The forest itself consists from 100 decision tree ...
jjepsuomi's user avatar
  • 111
1 vote
0 answers
171 views

Feature importance using random forest vs. SHAP

I recently came across SHAP while looking for feature-importance methods. To use SHAP, first a model needs to be created, and then based on the predictions made by the model, SHAP values are ...
lsr729's user avatar
  • 183
1 vote
1 answer
181 views

Can we add baseline to SHAP?

I have a doubt. I am currently using an integrated gradient for the DNN model for explainability. In that, we can specify the baseline as a parameter to the function. I am using all zeros for this. I ...
Pritam Sinha's user avatar
1 vote
0 answers
100 views

references on how to use shap values without the shap package

I am familiar with the shap python package and how to use it, I also have a pretty good idea about shap values in general, but it is still new to me. What I'm requesting are references (ideally python ...
Phillip Maire's user avatar
1 vote
0 answers
246 views

Do monotonic constraints prevent an XGboost to capture non-linear relationships in the data?

I have trained an XGBoost model (for a binary classification problem) and I have tested two scenarios: Scenario 1 - No Monotonic Constrained applied In this case I get a Gini on the training sample of ...
Giampaolo Levorato's user avatar
1 vote
0 answers
306 views

Using SHAP values as features in a classification problem

I'm looking for feedback on a methodology I've tried that has yielded strange results. Problem background: supervised multi-class classification problem for which I've used a random forest to create ...
jda's user avatar
  • 111
1 vote
0 answers
264 views

Samples to use when calculating SHAP values

I'm new to data science and I'm learning about SHAP values to explain how a Random Forest model works. I have an existing RF model that was trained on tens of millions of samples over a few hundred ...
Rayne's user avatar
  • 131
1 vote
1 answer
347 views

Shapley value, conditional expectation vs reference point

In Shapley, the marginal contribution of a feature is computed by comparing the performance of a model with and without a feature over all possible subsets of features. A common choice is using the ...
zzzbob's user avatar
  • 45
1 vote
0 answers
74 views

How marginal contributions of adding a variable in a model is calculated in determining SHAP feature importance?

I was trying to find feature importance using SHAP values in python for Isolation Forest. SHAP calculates the feature importance of a feature($i$) pertaining to a model($f$) for a datapoint($x$) using ...
aishik roy chaudhury's user avatar
1 vote
0 answers
310 views

SHAP for Deep Neural Network taking long time

I have 60,000 samples with each having 1,800 features. I have made a multilayer perceptron in Keras and I want to use SHAP values to arrive at global feature importance. Is the matrix too big for SHAP?...
Kamyar Yazdani's user avatar
1 vote
0 answers
46 views

not getting point predictions from sagemaker clarify [closed]

I wanted to create shap values for my predictions in sagemaker. I found out that I can use "clarify" functionality in sagemaker to get shap values. However, I want to get point predictions ...
Fatima's user avatar
  • 71
1 vote
0 answers
52 views

Passing reduced/different feature data to LimeTabularExplainer compared to the original model

I am trying to use LimeTabularExplainer class and explain_instance function to find explainations of my LightGbm (lgb) model. However, the lgb model uses complex feature set which are not ...
the M's user avatar
  • 11
1 vote
0 answers
62 views

Can we use Shap to interpret output changes?

Can we calculate the difference between Shapley values to interpret changes in the output? More precisely, if we get Shapley values for two different inputs, can we compare them to understand how much ...
giogix's user avatar
  • 163
1 vote
0 answers
146 views

Difference between shap values and feature contributions

I always found both concepts a bit confusing since they are quite similar. Would someone provide clear example where to apply each? Shap values ref: https://towardsdatascience.com/explain-your-model-...
room13's user avatar
  • 133
1 vote
0 answers
439 views

Tree Path Dependent expected value

I stuck on this topic for couple days and seems I need your help to understand what is expected value in TreeExplainer when feature_perturbation = tree_path_dependent, precisely what is expected value ...
jacekblaz's user avatar
0 votes
1 answer
116 views

Determining SHAP values for combined features

Say I have a dataset with three features: X1, X2 and X1 + X2. How would I determine the SHAP values for just singular features? I'll denote I denote the SHAP value of say X1 as SHAP(X1). Would it be ...
user156361's user avatar
0 votes
1 answer
28 views

SHAP values are explaining the wrong output value

I was checking the local accuracy property of the SHAP values. It states that for a data point $(X,y)$, the SHAP values $(s_1,s_2,s_3,...)$ of features $(x_1,x_2,x_3,...)$ sum up to the difference of ...
Abhay Gupta's user avatar
0 votes
1 answer
254 views

Why does the SHAP library in python seem to be so poorly coded/maintained, and how do I get a working environment?

I suppose this is a general question involving dependency conflicts. Without fail, every time I attempt to include the shap library into my project to examine explainability, it creates python ...
schmixi's user avatar
  • 21
0 votes
1 answer
196 views

Can using the mean of absolute Shapely Values for feature importance give very wrong results?

In a classification problem, suppose a model has 2 variables, A and B, and the null model (the model without any variable) predicts 50% probability for belonging to class 1 for all the instances. Now ...
Amir Jalilifard's user avatar
0 votes
1 answer
860 views

Shapley Values - How to interpret each value for each feature for a specific instance?

I am using Shap Values(the 'shap' module in python) to help me understand a bit better the relation between my features and my target. I am currently working on a binary classification problem. I know ...
Gabriel Monteiro's user avatar