On a neural network, how can I find out what features most contribute for some label (the dependent variable y)?


I have good news and bad news.

The bad news is that due to the nonlinear nature of neural networks, the same feature will contribute differently depending on the other features.

Here's a simple nonlinear function: f(a,b,c) = y = ba^3 + bc + c. One could argue that 'a' contributes most to this function, but when b is negative, a more positive a might make things worse. When b is zero, c contributes most. When b is negative, a very negative a will contribute most.

The good news is that you may be able to visualize some simple single-layer networks. There's a fun trick for this used in deep learning.

Assuming you've got x for your examples, W for your weights, and y for your outcomes, set the feature you want to show in y to 1 and the rest to zero, then multiply by W-transpose. You should get some output values for x which show you how connected everything is. It's not a perfect solution, but if you get some huge values coming out after the multiplication, you can say those values are highly-connected for y. (Usually.)

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.