Suppose I implement a supervised learning version of LSTM similar to this. Namely, I have these univariate time series data:
t Y
1 101
2 105
3 108
4 104
5 110
6 112
7 119
8 111
9 113
10 115
So Y at time 1 is 101, Y at time 2 is 105, etc.
The goal is to predict Y at time t+1 given Y at time t and t-1. That is, I can rearrange this data set to look like supervised learning:
Y_(t-1) Y_t Y_(t+1)
101 105 108
105 108 104
108 104 110
104 110 112
110 112 119
112 119 111
119 111 113
111 113 115
Notice here that each Y value from t=3 to t=10 (max) gets predicted only once.
Now say I partitioned my data set into overlapping minibatches of size 5. We'll consider only the first two minibatches for simplicity:
t Y
1 101
2 105
3 108
4 104
5 110
2 105
3 108
4 104
5 110
6 112
I do the same trick of converting this to a supervised learning problem for each minibatch:
Y_(t-1) Y_t Y_(t+1)
101 105 108
105 108 104
108 104 110
Y_(t-1) Y_t Y_(t+1)
105 108 104
108 104 110
104 110 112
Notice: now two of the predictions from the first minibatch are repeated in the second minibatch.
I know that if I had wanted to have only one prediction per Y value, then I should have converted the entire dataset to supervised first and then created minibatches.
Questions:
Is this alternative approach incorrect? If so, why? If not, should I average the predictions, or is there another accepted way of combining the predictions from several time steps like this?
EDIT: I should mention that the learned parameters of the model are different for each position, even though the data points are the same. For example, the parameters that would be used to predict Y_(t+1) = 104 in minibatch 1 are different from the parameters that would be used to predict Y_(t+1) = 104 in minibatch 2.
EDIT 2: I'm thinking this approach isn't incorrect, just perhaps a little odd. It would be similar, in a way, to a bidirectional LSTM (reading the input backward and forward to increase the exposure of the network to the end points - i.e. boost the signal). This also boosts the signal, but perhaps increases the bias by quite a bit. Thoughts?