4
$\begingroup$

In stacked LSTM, for example: 2 LSTM layers, LSTM_1 in order to pass the output of every time step to LSTM_2, so it needs to return hidden state value in every time step, like the architecture I drew below:

pic

But the picture above only has one unit, so I am wondering if it has more than one unit, for example memory unit=2, what will this model look like.

pic

or

pic

or can someone point out the wrong part, or give a sample of visualize architecture of LSTM model with multiple units, thanks!


perhaps my description was not clear enough, so i try to give a more clear description and graph of the question: I try to illustrate the question as the picture below, and my question is: is the architect correct when LSTM model has 2_units,2_layers.

enter image description here

$\endgroup$
4
$\begingroup$

The last image that you draw, where (at a given timestep) each cell of the second layer receives input from all cells in the first layer, is the right one.

You can think of recurrent layers as fully connected layers which receive sequences of input and apply their transformations at each timestep. They just receive other inputs depending on their previous inputs but the connectivity between layer is exactly the same as a fully connected : each unit of a given layer performs a weighted average of the activations of all the units in the previous layer (or of the inputs if it's the first layer).

| improve this answer | |
$\endgroup$
  • $\begingroup$ What would be the output if the 2 unit LSTM layer was followed by the dense (output) layer with just 1 unit? Would the output still contain A and B like in the third image? $\endgroup$ – Foshiba Sep 10 '18 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.