Modelling Unevenly Spaced Time Series

Question

I have a continuous variable, sampled over a period of a year at irregular intervals. Some days have more than one observation per hour, while other periods have nothing for days. This makes it particularly difficult to detect patterns in the time series, because some months (for instance October) are highly sampled, while others are not.

My question is what would be the best approach to model this time series?

• I believe most time series analysis techniques (like ARMA) need a fixed frequency. I could aggregate the data, in order to have a constant sample or choose a sub-set of the data that is very detailed. With both options I would be missing some information from the original dataset, that could unveil distinct patterns.
• Instead of decomposing the series in cycles, I could feed the model with the entire dataset and expect it to pick up the patterns. For instance, I transformed the hour, weekday and month in categorical variables and tried a multiple regression with good results (R2=0.71)

I have the idea that machine learning techniques such as ANN can also pick these patterns from uneven time series, but I was wondering if anybody has tried that, and could provide me some advice about the best way of representing time patterns in a Neural network.

Answer 1

ARIMA, Exponential Smoothing and others indeed require evenly spaced sample points. As you write, you could bucketize your data (say into days), but as you also write, you would lose information. In addition, you may end up with missing values, so you would need to impute, since ARIMA is not very good at handling missing values.

One alternative, as you again write, is to feed time dummies into a regression framework. I personally do not really like categorical dummies, because this implies a sharp cutoff between neighboring categories. This is usually not very natural. So I would rather look at periodic splines with different periodicities. This approach has the advantage of dealing with your uneven sampling and also with missing values.

Be very careful about interpreting $R^2$. In-sample fit is notoriously misleading as a measure of out-of-sample forecast accuracy (see here). I would argue that this disconnect between in-sample fit and out-of-sample forecast accuracy also means that there is no connection between in-sample fit and how well a model "understood" the data, even if your interest lies not in forecasting, but only in modeling per se. My philosophy is that if you can't forecast a time series well, you haven't understood it in any meaningful sense.

Finally, don't overdo the modeling. Just from eyeballing your data, it is obvious that something happened in June, on one day in August and in September/October. I suggest you first find out what this something was and include this in your model, e.g., as explanatory variables (which you can include in ARIMAX if you want to). What happened there is obviously not seasonality.

Answer 2

Since your question and the nice answer from @Stephan Kolassa discuss ARIMA and neural networks in particular, i wanted to mention that you can give the forecast package in R a go - it has a nnetar function that trains a simple feed forward neural network with 1 hidden layer and lagged inputs.

Maybe you could try something along the lines of:

• extract many features for each of your observations like day of week, day of month, weekday/weekend etc. (only datetime is mentioned as a potential dependency in your question so that's why i included this - but you can include all possible things that you believe could be influencing your variable of interest).
• lagged values of your variable of interest as well as the datetime information (like the day of week etc.) would be your inputs. you can include the datetime variables as external regressors (xreg) for example.

and predict the future values of your var of interest based on these inputs. Additionally, you could also think of including the the observed mean and variance/deviation on each given day of the value you want to predict. This would mean that you would have to first forecast your expected mean and variance with e.g. ARIMA then add that as additional input to the approach mentioned above.

hth.

Answer 3

Unevenly spaced / irregular times series is totally different from (regular) time series analysis. In fact, in my knowledge there is no perfect statistical model for it. There are some packages in R like ust and few more.