What's the intuition behind the hidden states of RNN/LSTM? Are they similar to the hidden states of HMM (Hidden Markov Model)? Thanks!
$\begingroup$
$\endgroup$
Add a comment
|
$\begingroup$
$\endgroup$
I personally don't think they are comparable to the hidden state of a Markov model. One key difference is that, in a HMM you can explain what a given state means to someone, where in a RNN/LSTM you cannot interpret a given state.
The closest thing that you can compare the hidden state of an RNN/LSTM is to think of it as the output of an intermediate layer of a fully-connected neural network but for time-series data.
And the larger the hidden state the more memory it can retain of the past.
