Is an LSTM cell autoregressive?

Question

I am currently writing some stuff up on Long Short-term Memory cells (LSTM), and stumbled over a question which I had trouble answering on the fly. The LSTM takes as input $h_{t-1}$ (besides the cell state and the local input $x_t$), which is also the output of the previous time step $t-1$. I was looking for a definition of the term autoregressive, because in the context of ML, it is often used for models which make a prediction $y_t$ while knowing about all prior predictions $y_{t-n}$.

Wikipedia states that

The autoregressive model specifies that the output variable depends linearly on its own previous values and on a stochastic term (an imperfectly predictable term)

If I understand correctly, I would argue that the LSTM is a first-order autoregressive model, because it sees exactly one value emitted in the past. At the same time, I think it is not autoregressive, because (if I understand the definition correctly) it does not exclusively rely on previous time steps, but also receives local information in the form of $x_t$. Is anyone able to clarify whether the LSTM cell could be considered autoregressive or not?

Answer 1

It is autoregressive because its output at time $t - 1$, $h_{t - 1}$, is received as input for the computation at time $t$ and used to generate $h_t$. The fact that there are other inputs like $x_t$ does not make it not autoregressive.

In some configurations (e.g. language models), its predictions are even used as input $x$.