2
$\begingroup$

I am building a time-series forecasting model to predict some patterns in climatological data.

The dataset consists of many (2 mln) time series which look for example as:

Example of one of the time series

However the observations all of these time series is unequally distributed (growing trend with years).

Distribution of observations for the time series shown above

Although I am still considering my approach (LSTM, exponential smoothing, etc.), I will have to deal with this unequal distribution of observations. Is there a golden standard for equalizing time series observations?

$\endgroup$

2 Answers 2

1
$\begingroup$

"Is there a golden standard for equalizing time series observations?" Ans.: No. But there are some approaches.

A time-series where the events occur in arbitrary time can be approached modeling the distribution of the number of events occcuring in a given time interval and the distribution of time intervals between the events. This is a point process.

Another approach is you resample the time-domain aggregating the values in fixed intervals of time, and look for this new time-series for make your forecast. Maybe the new time-series will be "well-behaved", enabling that statistical methods such as linear models to be applied directly.

Your approach with LSTM can be a good choice, because the pre-requisites of this machine learning algorithm are less restrictive (compared to ARIMA, e.g.). There many studies about combination of methods for this type of problem. A good reference about point process in this context is https://books.google.com.br/books?id=eMuCDwAAQBAJ&hl=pt-BR.

Another good reference about point process, with Python, is this Github: https://github.com/MatthewDaws/PointProcesses They have some Jupyter notebooks, as this: https://nbviewer.org/github/MatthewDaws/PointProcesses/blob/master/Temporal%20points%20processes.ipynb

$\endgroup$
0
$\begingroup$

If you take a Bayesian approach, you do not need to equalize the time points. The model would have greater confidence in predictions for the time points where there is more data.

Looking at the data, you have very few observations (less than 10 for several years). This limits the type of techniques you can use. It might not be appropriate to use machine learning or deep learning.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.