I am often confused about the lstm with more than one layer.
Imagine i have two lstm layer with 3 cells each layer.
What is exactly the input to the second lstm layer ?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
12
The input to the second LSTM layer is the output at each time step of the first LSTM layer.
-
$\begingroup$ But what exactly is the output of a lstm layer ? is it $h$ for each timestep ? $\endgroup$ Feb 28 at 11:12
-
$\begingroup$ Yes, the output of the first LSTM layer is the hidden state at each time step. $\endgroup$– noeFeb 28 at 11:17
-
$\begingroup$ ah and these "$ht$ are then like the $x$ in the second lstm layer ? $\endgroup$ Feb 28 at 11:21
-
$\begingroup$ What are "these $h_t$" and "the x"? I understand that you are looking at a picture but you did not share it so I can't possibly know what you are talking about. $\endgroup$– noeFeb 28 at 11:31
-
$\begingroup$ i can explain it more detailed. So if i have stacked lstm layers, then i know that the input to the second lstm layer are my output values okay so far so good. But if these outputs are the next input then what is the hidden state in the second lstm ? I mean every lstm layer has input values and a beginning hidden state? what would be the beggining hidden state of the second lstm layer for the first timestep ? $\endgroup$ Feb 28 at 11:42