Should I shuffle my `train_test_split` if my time series contains lagged features?

Question

I understand that it is not recommended to shuffle your training and test sets for time series, else the model will not be able to understand the time dependency of the features.

However, I am now using lagged variables for all of my features for my data frame. If I have a 7 day lag for each feature, the model, in this case a Random Forest (RF) has access to the past 7 days of each feature to predict each $\hat{y}_{i}$.

When using sklearn.model_selection.train_test_split can I set my shuffle=True? I have tested the model with a 7 day lag with/without shuffle and it overfits considerably when shuffle=False. The RF performs far better with shuffle=True, with my train/test $MAE$ converging well.

Is there anything wrong setting my shuffle=True when using time-lagged variables for time-series data?

Answer 1

Yes it is wrong to set shuffle=True.

By shuffling the data you allow your model to learn properties of the data distribution that might appear only in the test time periods. For example, if you have a trend in the data, shuffling will 'help' you handle it.

In a real-time scenario, you'll never have access to those properties of the distribution.