1
$\begingroup$

In several sources the marginal contribution is defined as

the difference between the prediction with and without the feature.

However, recently I read an article where the marginal contribution was defined as the:

difference between the feature effect minus the average effect.

Suppose a linear model:

fˆ(x) = β0 + β1x1 + … + βpxp

which is what a model prediction looks like for one data instance.

where x is the instance for which we want to compute the contributions. Each xj is a feature value, with j = 1,…,p.

The βj is the weight corresponding to feature j.

Therefore, the marginal contribution of feature j in the model is:

ϕj(fˆ)=βjxj − E(βjXj)

where E( β j X j ) is the mean effect estimate for feature j.

I am having difficulty to understand this definition. I would like to know where this definition come from and how it relates with the other definition:

marginal contribution is the difference between the prediction with and without the feature.

$\endgroup$

1 Answer 1

1
$\begingroup$

Here is a crucial paper to be read that I was also unaware of.$^\dagger$ The article points out the essential differences, both from a theoretical and empirical point of view, between feature importance scores that explain the model and those that explain the data.

Your first definition reflects the importance of a feature for a single prediction (i.e., local), while the second reflects the (entire) model's performance (i.e., global).


$^\dagger$ Thank you for that.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.