How to split a dataset into train and test sets for time series (multiple step-multiple output forecasting)?

Question

I am trying to use an LSTM neural net to do multiple step / multiple output forecasting (I predict multiple values in one time knowing some values in the past). But, I have realized that I must be careful when splitting my univariate series into training and test sets. In this post, I am going to share what must be done in my opinion, but I really would like to have yours.

• $(X_k)_{1\leq k\leq N}$ denotes my univariate time serie.

So basically, multiple output models are complex as they are trained to learn the dependencies between inputs and outputs as well as between outputs. This can be a problem as I transform my time serie like that :

$ D = (x=(X_k, ..., X_{k+p-1}) , y=(X_{k+p},..., X_{k+p+n-1}))_k $ with $p$ my number of steps and $n$ my number of outputs ($1 \leq k\leq N + 1 - (n+p)$). I want to split it into training and test sets. So, I must get :

$ D_{\text{train}} = (x=(X_j, ..., X_{j+p-1}) , y=(X_{j+p},..., X_{j+p+n-1}))_j $ where $j \leq N - 2(n+1) - p$

$ D_{\text{test}} = (x=(X_i, ..., X_{i+p-1}) , y=(X_{i+p},..., X_{i+p+n-1}))_i $ where $i+p+n-1=N$

In this case, I can avoid overfitting due to a naive splitting of my dataset $D = D_{\text{train}} + D_{\text{test}}$ where $+$ denotes a concatenation operator (preserving the timeline). Indeed, this would lead to over-fitting in my opinion as I would have in my train some part of my test.

For a better understanding, let's take an example : My univariate series is denoted as $(X_1,...,X_6)$. I want to predict $n=2$ values in the future knowing $p=2$ values of the nearest past. So, I transform my dataset as:

$ D = (x=[(X_1,X_2),(X_2,X_3), (X_3,X_4)] , y=[(X_3,X_4),(X_4,X_5), (X_5,X_6)])$

I split it naively, I get:

$ D_{\text{train}} = (x=[(X_1,X_2),(X_2,X_3)] , y=[(X_3,X_4)])$

$ D_{\text{test}} = (x=[(X_3,X_4)], y=[(X_4, X_5)]) $

I am afraid that it will lead to overfitting as the data $(X_3, X_4)$ is in both datasets. Maybe I am wrong, that's I want to know :)

Would you like to tell me if I am wrong and what do you think about? Thanks.

** Edits are written in bold **

Answer 1

Assuming you have sales data for 36 month and you want to forecast for next 6 or 9 months. Lets create the train and test data at 70 and 30.

train data= 36*70/100= 26(rounding value of 25.2) test data= 10(leftout sample)

Now we'll build a model on train data and forecast values on test data(those 10 points), to measure model performance we use MAE or MSE, always MAE or MSE values should be less. Once you model is doing good then forecast for future 6 to 9 months. i.e multiple output in one time prediction.