15
$\begingroup$

I'm trying to find an equivalent of Hinton Diagrams for multilayer networks to plot the weights during training.

The trained network is somewhat similar to a Deep SRN, i.e. it has a high number of multiple weight matrices which would make the simultaneous plot of several Hinton Diagrams visually confusing.

Does anyone know of a good way to visualize the weight update process for recurrent networks with multiple layers?

I haven't found much papers on the topic. I was thinking to display time-related information on the weights per layer instead if I can't come up with something. E.g. the weight-delta over time for each layer (omitting the use of every single connection). PCA is another possibility, though I'd like to not produce much additional computations, since the visualization is done online during training.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
10
$\begingroup$

The closes thing I know is ConvNetJS:

ConvNetJS is a Javascript library for training Deep Learning models (mainly Neural Networks) entirely in your browser. Open a tab and you're training. No software requirements, no compilers, no installations, no GPUs, no sweat.

Demos on this site plot weighs and how do they change with time (bear in mind, its many parameters, as practical networks do have a lot of neurons). Moreover, if you are not satisfied with their plotting, there is access to networks parameters and you can plot as you wish (since it is JavaScript).

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks! Interestingly enough they chose to use multiple Hinton Diagrams to plot their weights. I still think it's hard to interpret as soon as you have too many layers/connections but it's good to see it in action at least. $\endgroup$
    – runDOSrun
    Commented Dec 10, 2014 at 20:52
5
$\begingroup$

Based on my cursory understanding of the topics, associated with your question, I think that Gephi (the original gephi.org link redirects there) should be able to handle neural network dynamic visualization. It seems that, in order to achieve your goal, you need to stream your graph(s) with corresponding weights. For streaming, you most likely will need this plug-in.

UPDATE: You may also find useful SoNIA software.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Very interesting idea! Really, visualizing a deep net like a social network is something I've not thought about. The main difference between the models is that these graphs code information in their nodes while neural networks do it within their connections. But it could be modified, e.g. by setting the social network node values to the outgoing connections weights of the neural network. $\endgroup$
    – runDOSrun
    Commented Dec 11, 2014 at 12:49
  • $\begingroup$ I'm glad that you like the idea. Feel free to upvote/accept. And don't forget to review SoNIA software, with link to which I recently updated my answer. Finally, if you use (or plan to use) R, here's another relevant interesting info for you: sna.stanford.edu/rlabs.php. $\endgroup$
    – Aleksandr Blekh
    Commented Dec 11, 2014 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.