0
$\begingroup$

I have a dataset consisting of multiple timeseries for multiple users. So per user I have multiple timesteps, a value to predict per timestep and a list of features per timestep. I am currently employing an LSTM model, but I am not completely sure on how to split the dataset into a train/validation/test set. I can think of two options

  • Split based on users. So train on a few users, then test on a few different users.
  • Remove the last few timesteps of all the timeseries. So train on the first x % of timesteps of each user, and then test on the remaining timesteps.

To me the first method seems fairer, as this is completely unseen data. But the second method seems more inline to a real life scenario, in which you already have a timeseries for a user and want to predict the next timestep. But I'm afraid there could be some bias to later timesteps. If all later timesteps converge to some value, the model would automatically predict more accurately.

Could somebody give me some pointers in the right direction, or show me some related literature? Thanks!

$\endgroup$

1 Answer 1

0
$\begingroup$

If the time series of the individual users are correlated in any, even a subtle way, if your model has sufficient capacity it will definitely exploit the info from already seen future timesteps. So, unless your timeseries are not 100% uncorrelated, I would strongly advice against feeding in per-user, and rather do the time dimension split. Given you are putting these users in a joint dataset, I assume its very likely that there is a correlation between their timeseries. So, try the time split; you can read about walkforward testing, as a keyword.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.