Long range forecasting with sequence-to-sequence models

Question

I have a task where I want to forecast daily observations for 1 year or 2 years in advance at multiple locations--so 365 or 730 days in advance. I actually have a pretty good dataset, meaning daily observations going back 40 years at those sites. So I have a lot of data to train with. I am working with climate and geologic data, so the time series is strongly cyclical or periodic with some random variation.

I was trying to use a Sequence-to-Sequence LSTM network with attention to start with. I have 365 days input and trying to predict 365 days output. I am just starting to collect the results so far. But I was wondering if there are any specific models designed for long-range time-series forecasting? I thought sequence to sequence models still only worked well for say 20-30 day output, but I was not sure if there has been any developments on that front.

Thanks.

Answer 1

I am wondering why you chose a Neural Network approach in the first place. In general a recurrent Neural Network requires many more parameters to be fitted than for instance a SARIMA model. Therefore, they are more sensitive to spurious artefacts in the train sample. (This is the price you pay for being more flexible in the way the relationships between past and future values are modelled). Given the fact that you mentioned the strong periodicity in your data I think modelling the data using a SARIMA model or even a model with seasonal dummies may generalise much better out of sample, given the sample size that you have. In addition you could add a cyclical trend component to also incorporate other cyclical components in the data. Hope this helps. Both the SARIMA model and the addition of a cyclical component are often described in the econometric literature. Comparing your current model with a SARIMA with: {p=0; d=0; q=0; P=1; D=1; Q=1, s=365 and trend='c'} may be a good place to start (These arguments refer to the statsmodels SARIMA that can be used with Python "statsmodels.tsa.statespace.sarimax.SARIMAX").