LSTM - How many times should I look back to predict next six hours -Multivariate Time-Series

Question

I am still finding confusing on look back topic when using LSTM for time-series analysis. If I have hourly data and I want to predict next 6 hours with multiple predictors, should I look back up to 6 hours when I prepare my training set?

OR

Should I look back one hour (shift 1) and then predict next hour and take that predicted value and feed it back to predict the value after until next 6 hours?

This concept is still a little fuzzy for me in LSTM. Any thoughts would be appreciated.

Answer 1

The second option sounds fun but better don't do it: errors will sum up and it's much harder to track where errors come from. Like you have some error for the 1st hour, feed it back in the net, predict the 2nd hour get an error there and can't really tell how much of that is caused by the error of the first hour.

Besides that, I haven't seen any application which works with this feedback mechanism.

Answer 2

So at time $t$ we want to predict values $y_{t+1}$ up to $y_{t+6}$ correct? You should use $x_0$ up to $x_t$ as inputs and use 6 values as your target/output. Then when you get new information, you add $x_{t+1}$ and use it to update your cell state and hidden state of your LSTM and get new outputs. The problem with feeding predictions is that errors will accumulate, although this does happen with sequence generation models (like seq2seq). What you would do then is during training use two types of training, one where you use current predictions and one where you use your known training data, both a fraction of the time. This is called 'teacher forcing'.

If we use my first suggestion, here is an explanation how to prepare your data. To train your model you will need to use lags to prepare your targets. Let's say we have $x$ as one-dimensional and we are trying to predict the next 6 values of $x$, you would get as $x$ and $y$:

| t | x   | y                     |
| 0 | 0   | [-1, 1, 2, 3, 2, 1]   |
| 1 | -1  | [1, 2, 3, 2, 1, 0]    |
| 2 | 1   | [2, 3, 2, 1, 0, -1]   |
| 3 | 2   | [3, 2, 1, 0, -1, -2]  |
| 4 | 3   | NaN                   |
| 5 | 2   | NaN                   |
| 6 | 1   | NaN                   |
| 7 | 0   | NaN                   |
| 8 | -1  | NaN                   |
| 9 | -2  | NaN                   |

The last six steps don't have a target because it's not available in our training set. The loss function would be a (weighted) mean squared error over the six predictions per time step. If you really don't want to throwaway data, you could make a custom loss function that can use the fact that only 4 targets are available because we are near the end, but that will complicate things.

EDIT: With regards to the question in the title, you would like to look back as far as possible, which basically means you always start at $t=0$.