Does shuffling data for time series forecasting help?

Question

So I am trying time series forecasting using LSTM's. The aim is to predict $Y$ given $X$ using regression.

I had already converted the input data into a sliding window format such that if my input data was of the form:

X = [x0, x1, x2,.....]
Y = [y0, y1, y2,.....]

Then I converted it into:

Xnew = [(x0, x1, x2), (x1, x2, x3), (x2, x3, x4),...]
Y    = [         y2,           y3,           y4,...]

Still, upon training my data I find a very high validation_loss.

Since validation_split takes only the fraction of the data from the end, I thought maybe I should try and randomize the data before training it. However, in that way, will time series have any meaning?

I found a similar question, but I had apparently already tried what was suggested: Is it valid to shuffle time-series data for a prediction task?

Answer 1

Shuffling data would not seem to make sense here, since your model has "memory". You're not predicting $y_i$ from only $x_i$, but also $x_{i-1}$ and $x_{i-2}$. If you shuffle the data and perform prediction, you are implying that $x_1, x_2, x_3$ should give the same value as $x_2, x_1, x_3$ or $x_9, x_5, x_3$, or any series of values that merely ends in $x_3$ (since the target value is always $y_3$, regardless of the other $x$). If your target value actually does depend on preceding variables, shuffling the data breaks that relationship. If it does not depend on preceding values, it's arguably not a time-series model, since the ordering of observations is irrelevant.

Answer 2

In RNN sequence is utmost important. So strict no for shuffling as it will break the sequence.

Though I think, it will be good if we can shuffle different batch series, such that if a particular series used in batch i, next time that series can be the part of some different batch say j.

If you try, please share your results.

Answer 3

LSTMs have memory, so it matters in what order the model sees your samples. From the answer you linked:

The model's internal parameters are changing and persisting with each new example it sees. The current prediction depends on the last prediction. Recurrent neural networks have memory, so order matters.

If you're worried that your estimate of loss is inaccurate, you can obtain a more stable estimate of validation_loss by using time-series cross-validation.

If you are instead worried that the performance of your model is insufficient, keep in mind that hyperparameter tuning will likely be important for this problem. No need to fix the size of the historical window at 3; perhaps a larger window size is better? Other hyperparameters such as the size of the hidden state and the number of recurrent layers can also have a big impact on performance.