Confusion regarding what constitutes a feature in a LSTM?

Question

I have a Time Series problem, where I am trying to predict a single output at time $t$, $y_t$, given the $2$ previous time steps; $X_{t-2}, X_{t-1}$.

Let's just look at one observation for simplicity.

At a given time step $t$, I have $3$ features and a single output. Let's say $[a_t, b_t, c_t, y_t]$, where $a_t, b_t, c_t$ are my features, and $y_t$ is my output (the value I want to predict).

So, If I want to predict $y_t$ given the previous $2$ timesteps, this would look like

$$[ [a_{t-2}, b_{t-2}, c_{t-2}, y_{t-2}],\\ [a_{t-1}, b_{t-1}, c_{t-1}, y_{t-1}], \\ [a_{t}, b_{t}, c_{t}, ?]]$$

I don't have a value for $y_t$ here, and I need to pass in $4$ features to my $X_t$, so how does this work exactly?

At time $t$, I am again aware of my features $a_t, b_t, c_t$, and I want to predict $y_t$. But if I am only looking at the previous 2 timesteps here, I don't understand how the LSTM knows anything about the features at the current time step?

Answer 1

The LSTM model doesn't know anything about the features at the current time step. Since this is a prediction model, the current timestep is the thing you are trying to predict, and therefore all of your inputs must be from a previous timestep. The correct representation would be:

[[$a_{t-2}$, $b_{t-2}$, $c_{t-2}$, $y_{t-1}$],

[$a_{t-1}$, $b_{t-1}$, $c_{t-1}$, $y_t$]]

Where $y_t$ is the thing you are trying to predict.

If you wanted your LSTM model to make judgements directly based on the previous output, you would have to add that as a feature.

[$a_{t-1}$, $b_{t-1}$, $c_{t-1}$, $y_{t-1}$, $y_t$]