1
$\begingroup$

I am using a recurrent network for time series forecasting.

The prediction from the last cell in the network always seems to be the most accurate. For example if I have 20 cells (so my input samples are of length 20), the prediction from the latter cells (e.g. cells 11-20) seem better than the predictions from the early cells (e.g. cells 1-10).

Is this normal?

Lets assume my network only has 3 cells. At present I would input data x(t=0, t+1, t+2) to obtain predictions for t=0, t+1, t+2.Is this the wrong approach? If I want good predictions at all timesteps, do I need to using a "moving window" i.e. should I use a rolling sequence of inputs:

For prediction at t=0, input x(t-2, t-1, t=0)
For prediction at t=1, input x(t-1, t=0, t=1)
For prediction at t=2, input x(t=0, t+1, t+2)
$\endgroup$
  • $\begingroup$ You should take care not to violate the nature of time-series predictions. How you describe your input/output data, it could be that in one single step, the network sees the data that it should be predicting. If you're unsure, check out an answer I posted on this topic a while back. $\endgroup$ – n1k31t4 May 21 '18 at 10:59
1
$\begingroup$

I am using a recurrent network for time series forecasting.

Interesting would be especially the type of the network you chose. How many layers? How many neurons per layer?

The prediction from the last cell in the network always seems to be the most accurate.

I assume by cells you refer to your samples. Always, if you learn in datastreams (online), the network in later iterations has seen more data than the network in early iterations. Thus, with having seen more samples, your predictions become more accurate. Of course, this assumes that you do online learning on your datastream.

At present I would input data x(t=0, t+1, t+2) to obtain predictions for t=0, t+1, t+2.Is this the wrong approach?

This depends on how your RNN is set up. A Jordan Network for example has context neurons, which should do exactly what you propose. The reason for recurrency in the network is typically that you want to simulate short term memory - meaning, in a mathematical view, you want to feed information from previous iterations into the front of the network. Thus, since this feature should be taken care of in your RNN, I do not think it is good to extend your input, since you introduce additional, potentially useless weights that have to be learned by the network.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.