LSTM: Taking previous output values as feature

Question

As far as I know, there is practically no limit on the number of dimensions of input feature for LSTM. And it apparently can learn the sequence of data.

My question is does LSTM by nature, also take previous output values besides feature vector as new feature element for the next output values? i.e. If we have:

[t] -> [x1, ..., xn] [y_t] [t+1] -> [x1, ..., xn] [y_(t+1)]

Is it necessary to manipulate data like this

[t] -> [x1, ..., xn, y_(t-1)] [y_t] [t+1] -> [x1, ..., xn, y_t] [y_(t+1)]

or LSTM is already handling this for us?

Answer 1

My question is does LSTM by nature, also take previous output values besides feature vector as new feature element for the next output values?

You didn't specify which framework you are using, so I will explain for a custom (c++ etc) implementation.

At each timestep LSTM will:

1. Take a new input from you (for example, as a one-hot encoded vector)
2. Internally fetch a separate vector which is the LSTM's output from Timestep [t-1]
3. These two vectors are like a 'fuel' and will be used to shape out a Cell at [t]
4. The result gets produced for this Timestep [t]

Back Propagation creates gradients for the weights within the LSTM, but also gradients for these two vectors.

Because there are multiple timesteps, you get a Gradient during each Timestep, which has to be added (component-wise) before being used.

Note, this last url link has a blue arrow at the bottom right of the screen, so you can click to see the next slides.

Answer 2

LSTMs handle this for you.

You can search for some tutorial on the LSTMs that explains how they works. One that I like is this.