0
$\begingroup$

I would like to try a technique I saw in an article a while ago where, in order to understand what each neuron is doing, you apply specific inputs to the net and see which one of them is most activated.

Is there a simple way to do it with Keras (preferably)? I can take the weights of each neuron after the NN has been trained and use forward (predict) functions to activate it, but how would I get the activation result of each individual neuron? Is there a function for that?

TensorFlow and PyTorch offer more low level functions so I could do the same thing if I programmed the NN almost from scratch, but even so, does anyone know a simpler way?

If you have tips for this task also, they would be more than welcome. I don't know if functions for this kind of task already exist, but, if they don't, I would say it would be worthwhile creating them, people on the market have complained often to me that the main reason why they are not yet using NNs is that they don't know how to understand the NN's thinking.

Improve this question
$\endgroup$
3
  • $\begingroup$ Could you give the link of this article in order to look at the algorithms and help you building it $\endgroup$
    – LaSul
    Jan 9, 2019 at 19:47
  • $\begingroup$ I don't know if it will be that helpful, they used CNNs instead of NNs, so it is much more complicated there. What I had in mind was just simple NNs. Anyway, the article is Visualizing and Understanding Convolutional Networks, by Zeiler and Fergus. $\endgroup$
    – Philippe Fanaro
    Jan 9, 2019 at 20:09
  • $\begingroup$ You can use tensorboard with keras, and get prettu much a deep insight about your network. Apart from that, here's a real fun tool to visualize a neural network in 3D: tensorspace.org/html/docs/startHello.html $\endgroup$
    – Syed Ali Hamza
    Jan 9, 2019 at 20:24

1 Answer 1

Reset to default
1
$\begingroup$

From Keras Issue #41:

One simple way to do it, is to use the weights of your model to build a new model that's truncated at the layer you want to read. Then you can run the ._predict(X_batch) method to get the activations for a batch of inputs.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Cool, I'll give it a try and see how it goes. I've also spent a whole day on learning PyTorch and finally managed to reverse engineer an NN. I'll see which method works best and come back to this post hopefully soon. $\endgroup$
    – Philippe Fanaro
    Jan 11, 2019 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.