In RNNs why do networks always use the last output vs the last input?

Question

All the descriptions of RNNs introduce some equation like:

$\ h_t = f_W(h_{t-1},x_t)$

and I'm wondering why we don't just go straight to the "source", ie the last input like :

$\ h_t = f_W(x_{t-1},x_t)$

The only thing I can maybe think of is maybe the RNN can store some sort of "state" in ht-1

But then, along the same vein, what would happen in an LSTM architecture if we used previous inputs vs previous outputs? It seems to me that the presence of the memory functionality should be able to avoid this potential problem (if it even is one)

Answer 1

That would work if the Markov Assumption was in place here (i.e. the last state was enough to determine the new state).

However, that is exactly what RNNs try to break. RNNs try to encode the whole history of inputs and predictions to determine the next output.

So in the formula, the $h_{t-1}$ is the last output, but it is a hidden state which encodes all the previous inputs, outputs, and calculations. So when you have new information coming in (i.e. $x_t$), your model can make use of all the history and the new input to determine the new input.

The same applies to LSTM, you are working with a sequence that the new output depends not only on the last input but on more things from the past, and this information is encoded in $h_{t-1}$.