I have time-series data for every single day from the last 5 years with seasonal variation and a general increase in trend. This is what my data looks like:

enter image description here

And I am trying to predict for every single day for 4-5 years in the future. Approaches I have used currently:

  • LSTM, GRU (but they are extremely prone to overfitting ad I am trying successive predictions which results in massive error accumulation and often flat over time for smaller lookbacks and sine wave-driven patterns for long lookbacks.
  • SARIMAX and other seasonal forecast models but tend to scale very bad for a large period of predictions. Mostly they eventually result in a curve that follows a trend(increase/decrease) but values are almost linear or unrealistically close to each other.

What I am currently thinking of:

Separate the same day of each year and fit a model to those values and then have successive predictions for the same day for subsequent years. Thus having 365 different models and then concatenating the resultant values to have the whole year and future years predictions. This way I'll perfectly preserve the seasonality and yet accumulation errors will be minimized. So my question is:

Is this a good idea (and if it is, is there a method to do it feasibly without assessing each of the 365 models) or I should try any other approach?

  • $\begingroup$ I don't think this would be a good idea. First of all, are you trying to do daily predictions for 4-5 years having 4 years of data? Do you have more? Also, what is the objective of the daily prediction? Do you really need those 4-5 years? Would not be better a monthly model for example and keep it updating? $\endgroup$
    – Let's try
    Aug 1, 2020 at 10:44
  • $\begingroup$ You could use MLforecast package. It takes care seasonality and you can add also regressors. github.com/Akai01/MLforecast $\endgroup$ Aug 5, 2020 at 13:25

1 Answer 1

  • Have you tried the good old Fourier Transform?
  • If you can take the load of math, you can try the Wavelet Transform, but first I would recommend a simple Fourier Transform.

A Guide


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.