Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
RNN or LSTM are known to hold the previous timestamp data as "memory" so that short or long range dependencies can be remembered.
But in the following simple keras model, where is that delay or memory thing? It is just taking each input at a time and throwing the output at same timestamp!
model = Sequential()
model.add(LSTM(10))
model.add(Dense(2))
The LSTM is just applied to each of the time steps in the input. The information from the distant time steps is passed from time step to time step in the hidden state of the LSTM. That's how it can use information from past timesteps in the current one.
Due to the way you initialized the LSTM, it just returns the output of the last time step. However, if you initialize the LSTM with the parameter return_sequences=True it will return the whole sequence of intermediate outputs instead of just the final one.
