Forecasting time series: Method Selection

Question

Im new to forecasting time series and Im looking for some advice on selecting the best method based on the analaysis of the graph. I have the following data and based on the little knowledge I have, Im assuming it is a stationary time series because of the shape of the data and because of the result I got when performing a Dicker Fulley Test

Test Statistic                -6.560544e+00
p-value                        8.402824e-09
#Lags Used                     2.100000e+01
Number of Observations Used    1.164000e+03
Critical Value (5%)           -2.864026e+00
Critical Value (1%)           -3.435980e+00
Critical Value (10%)          -2.568094e+00
dtype: float64

That made me choose Arima with a p and q values of 1. However the result Im getting are pretty awful. Can anyone guide on how should I choose the adequeate method for forecasting and if case ARIMA is a good choice, how should I tweak parameters to improve my results? The end goal is to predict the next month of data, so it might not be necessary to use all of the data.

Answer 1

You can try two different approaches:

1) Kalman filter, the method is battle-tested and has proven useful in many areas.

Resources:

• Understanding the Basis of the Kalman Filter
• kalman filter in pictures

2) Recurrent Neural Networks, the LSTM and GRU architectures are particularly interesting for time series predictions.

Resources:

• RNN effectiveness
• Understanding LSTMs

To do regression and predict future data points, you would need to build a training dataset consisting of a sequence of events. Let's say a value $x$ for every timestamp $t$.

Your data seems to have 1 dimension, so both the network input layer and the output layer would consist of 1 unit. You would then train your model to predict $(x_{t+1})$ given $(x_{t})$.

Let $M$ be our trained model and let's say you want to forecast a data point at time $k$ and you know the current value at time $t$.

$M(x_{t}) = (x_{t+1})$

$t = t+1$

$M(x_{t}) = (x_{t+1})$

$...$ increment $t$ and keep predicting until $t+1 = k-1$

$M(x_{t+1}) = (x_{k})$

Put in pictures this corresponds to:

(picture from Udacity lecture about Deep Learning)

Answer 2

I'm not very familiar with ARIMA but I've used for a very similar chart a RNN and I got good results. Look at GRU which are very efficient.