1
$\begingroup$

As the title says, let's assume we have some neural network. More specifically, a regression network in case that changes anything. It takes multiple 2D input channels. How could one measure how much each input channel is contributing to the accuracy of the network on a validation set, to know which channels to keep and which are not really necessary?

$\endgroup$
0
$\begingroup$

Neural networks are black boxes, and figuring out its internal workings can be really hard. When you decide to use ML for your model, you are sacrificing accuracy for interpretability.

However, that does not mean it is totally impossible to interpret networks. The most tedious way to determine feature importance would be to remove features and see how it impacts the model. If you're interested in research into the subject, I've encountered some research papers to use decision trees to understand networks.

https://openaccess.thecvf.com/content_CVPR_2019/papers/Zhang_Interpreting_CNNs_via_Decision_Trees_CVPR_2019_paper.pdf

https://arxiv.org/pdf/1802.02195.pdf

$\endgroup$
0
$\begingroup$

After searching for multiple options, I'd recomment using Captum. Implements multiple NN analysis papers to run on any pytorch model: https://captum.ai/

So it can easily calculate the imporance of features for the ouptut, a specific layer or even a specific neuron.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.