1
$\begingroup$

According the LSTM design:

enter image description here

The hidden state (ht) is output twice (1 and 2 in the picture).

  1. If they are the same, why we need them twice ?
  2. Is there a different use for each one of them ?
  3. According to

nn.lstm

there are 3 outputs (output, h_n, c_n). I didnt understand what is the different between output and h_n ? (Doesn't they need to be the same) ?

$\endgroup$

1 Answer 1

1
$\begingroup$

ht was initially defined as a differential function, which value is the same in output and in the next LSTM cell.

LSTM uses the previous steps in a sequential way and chooses whether to memorize or forget according to h(t-1) and C(t-1) and the inner weights, to set h(t) and C(t). h(t) is the cell's output, and it is sent to the next cell in order to keep a sequential logic.

It is quite complex to explain in a few words but let's say that the forget and memorize weights are set during the training process thanks to an auto-regulated system (named "the constant error carrousel") that takes into account several scenarios and at the same time avoids the neurons to diverge during training.

See main publication: https://www.researchgate.net/publication/13853244_Long_Short-term_Memory

Note: Google spent 10 years understanding LSTM's publication. It's very complex but very interesting.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.