7
$\begingroup$

I'm new to deep learning, I am learning LSTM for my PhD work. This is a simple LSTM network for sequence classification. This code is from MATLAB tutorial: layers = [sequenceInputLayer(1) lstmLayer(5,'OutputMode','last') fullyConnectedLayer(3) softmaxLayer classificationLayer];

For simplicity, the dimension of the input sequence is 1, there are 3 classes.

I am trying to draw the diagram of this network. This is my attempt: enter image description here

Is this right? Should the LSTM blue units be connected? The orange units are the softmax layer, is there any symbol the should be on each unit (like ∑)? The same question for each layer? Should be any extra layer to represent the "classificationLayer"? The fullyConnectedLayer is represented implicitly in the full connection in the last layer, Do I need to add any extra layer for this? Any other comments, please?

$\endgroup$
4
  • $\begingroup$ Chris Olah has some excellent LSTM diagrams here: colah.github.io/posts/2015-08-Understanding-LSTMs $\endgroup$
    – redhqs
    Jul 8 '18 at 10:58
  • 1
    $\begingroup$ That's right, I passed by this link before. But, it only illustrates the LSTM 'unit' diagram. I need to illustrate the whole layer and the whole network. $\endgroup$ Jul 8 '18 at 11:18
  • $\begingroup$ Were you able to find something? It's kind of crazy how hard this is to find. $\endgroup$ Nov 9 '19 at 21:29
  • $\begingroup$ Actually, I did not reach anything. So, I had to improvise. I made some simple scetch and put it in a paper. The paper was accepted and it is not published in an impacted journal. You can find it here: ieeexplore.ieee.org/abstract/document/8760508 Sorry, this site does not allow attaching figures to comments. $\endgroup$ Dec 1 '19 at 13:47
1
$\begingroup$

When thinking of rnn/lstm/gru layer keep few points in mind.

  1. What is your inputs size? In this case we have 5 word sentence so 5 input circle going into lstm layer with that every word value will be multiple by same weight value.
  2. How many lstm cell will be present inside lstm layer? To keep it simple as we have 5 words we are keeping 5 lstm cell for each word then carry forwarding it's memory into next lstm cell.
  3. How many output categories we have? The last lstm cell will be connected to the all 3 dense layer neurons and at each of the neurons softmax operation will happend just like normal fully connected neural network dense layer works.

Based on the network description lstm network

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.