1
$\begingroup$

Suppose we have a Neural Network with a binary output (0 or 1). What I am trying to do is to remove neurons or layers from the NN while maintaining a correct classification for all the instances that were classified as 1 in the original NN, same thing for the output 0. Said differently, is there any way to spot neurons that are paramount to the correct classification of the instances of a particular class ? The aim is to remove all the unnecessary neurons regarding that output. Currently, I am trying to use the back propagation phase to try to attribute a fitness to each neuron regarding its contribution to a certain class. In the case of Binary Neural Networks (binary weights and activations), a research track could be compiling the NN to a Boolean Formula and reasoning on it to spot the neurons that does not contribute to the chosen output, but it is not always obvious to carry out this compilation.

$\endgroup$
2
  • $\begingroup$ At what threshold? Your network is outputting a probability that you then round up it 1 or down to 0, depending on the side of the threshold (such as 0.5 or 0.7 or 0.05) it falls. $\endgroup$
    – Dave
    Commented Jun 3, 2020 at 11:16
  • $\begingroup$ How about Layer-wise Relevance Propagation (LRP)? $\endgroup$
    – bonfab
    Commented Jun 3, 2020 at 11:40

2 Answers 2

1
$\begingroup$

There is a lot of literature on this topic. For example, have a look at https://arxiv.org/pdf/1710.09282.pdf

$\endgroup$
1
$\begingroup$

This is a big field in deep learning. You're going to want to search on network pruning, which is a method of model compression.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.