I have a dataset consisting of multiple timeseries for multiple users. So per user I have multiple timesteps, a value to predict per timestep and a list of features per timestep. I am currently employing an LSTM model, but I am not completely sure on how to split the dataset into a train/validation/test set. I can think of two options
- Split based on users. So train on a few users, then test on a few different users.
- Remove the last few timesteps of all the timeseries. So train on the first x % of timesteps of each user, and then test on the remaining timesteps.
To me the first method seems fairer, as this is completely unseen data. But the second method seems more inline to a real life scenario, in which you already have a timeseries for a user and want to predict the next timestep. But I'm afraid there could be some bias to later timesteps. If all later timesteps converge to some value, the model would automatically predict more accurately.
Could somebody give me some pointers in the right direction, or show me some related literature? Thanks!