How to train-test split a timeseries?

Question

I have a dataset consisting of multiple timeseries for multiple users. So per user I have multiple timesteps, a value to predict per timestep and a list of features per timestep. I am currently employing an LSTM model, but I am not completely sure on how to split the dataset into a train/validation/test set. I can think of two options

• Split based on users. So train on a few users, then test on a few different users.
• Remove the last few timesteps of all the timeseries. So train on the first x % of timesteps of each user, and then test on the remaining timesteps.

To me the first method seems fairer, as this is completely unseen data. But the second method seems more inline to a real life scenario, in which you already have a timeseries for a user and want to predict the next timestep. But I'm afraid there could be some bias to later timesteps. If all later timesteps converge to some value, the model would automatically predict more accurately.

Could somebody give me some pointers in the right direction, or show me some related literature? Thanks!

Answer 1

If the time series of the individual users are correlated in any, even a subtle way, if your model has sufficient capacity it will definitely exploit the info from already seen future timesteps. So, unless your timeseries are not 100% uncorrelated, I would strongly advice against feeding in per-user, and rather do the time dimension split. Given you are putting these users in a joint dataset, I assume its very likely that there is a correlation between their timeseries. So, try the time split; you can read about walkforward testing, as a keyword.

Answer 2

For a single time series, I usually make the split in various train test splits ie:

• You have data from August of 2020 to August 2021.
• Split taking 2 months by 2 months, this process is called splitting window, then you have a 'window' of two months of data, based in this you can train, make the inference and calculate your metrics using the test set which will be taken the values after this 2 months.