My question is does LSTM by nature, also take previous output values besides feature vector as new feature element for the next output values?
You didn't specify which framework you are using, so I will explain for a custom (c++ etc) implementation.
LSTM will:
- Take a new input (for example, as a one-hot encoded vector)
- Take a separate vector which is the LSTM's output from Timestep [t-1]
- These two vectors are like a 'fuel' and will be used to shape out a Cell at [t]
- The result gets produced for this Timestep [t]
Back Propagation creates gradients for the weights within the LSTM, but also gradients for these two vectors.
Because there are multiple timesteps, you get a Gradient during each Timestep, which has to be added (component-wise) before being used.
