0
$\begingroup$

For neural network feature importance, can I zero-out all features except one in order to gauge that feature's importance? I know shuffling a feature is one approach.

For example, leaving in the 4th feature.

feature_4 = [
    [0.,0.,0.,1.15,0.]
    [0.,0.,0.,1.76,0.]
    [0.,0.,0.,2.31,0.]
    [0.,0.,0.,0.94,0.]
]

_, probabilities = model.predict(feature_4)

The non-linear output of activation functions worries me because activation of the whole is not equal to the sum of individual activations:

from scipy.special import expit #aka sigmoid

>>> expit(2.0)
0.8807970779778823

>>> expit(1.0)+expit(1.0)
1.4621171572600098

And softmax seems much less straightforward in comparison to sigmoid.

$\endgroup$
1
  • $\begingroup$ i suppose another reason to shuffle a column would be to preserve the interaction with other features $\endgroup$
    – Kalanos
    Dec 23 '21 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.