ARMA/ARIMA on energy forecasts timeseries: strange prediction

I'm trying to use ARMA/ARIMA with the statsmodel Python package, in order to predict the gas consumption. I tried with a dataset of this format:

Using only the gas column.

from pandas.tseries.offsets import *

arma_mod20 = sm.tsa.ARMA(januaryFeb[['gas [m3]']], (5,3)).fit()
predict_sunspots = arma_mod20.predict('2012-01-13', '2012-01-14', dynamic=True)
ax = januaryFeb.ix['2012-01-13 00:00:00':'2012-01-15 22:00:00']['gas [m3]'].plot(figsize=(12,8))
ax = predict_sunspots.plot(ax=ax, style='r--', label='Dynamic Prediction');
ax.legend();


Why is the prediction so bad?

I'm not an expert on time series, but I have a general advice: may I suggest you to try other packages (and various parameters) to see, if there are any differences in results.

Also, unless you have to use Python, I'd recommend to take a look at the R's extensive ecosystem for time series analysis: see http://www.statmethods.net/advstats/timeseries.html and http://cran.r-project.org/web/views/TimeSeries.html.

In particular, you may want to check the standard stats package (including functions arima() and arima0), as well as some other packages: FitARMA (http://cran.r-project.org/web/packages/FitARMA), forecast (http://cran.r-project.org/web/packages/forecast) and education-focused fArma (cran.r-project.org/web/packages/fArma), to mention just a few. I hope this is helpful.

• My vote is also for R. As far as I know, there is no time series decomposition functions in statsmodel (Python). Though in this case decomposition could be crucial to improving prediction. I see notable seasonal peaks. – sobach Jul 16 '14 at 8:45
• @sobach: Thank you for R solidarity. In regard to the rest of your comment, similarly to now famous tweet, I can neither confirm, nor deny that :-). [Since it's beyond my current level of knowledge on the subject.] – Aleksandr Blekh Jul 16 '14 at 10:38

Gas usage has a daily cycle but there are also secondary weekly and annual cycles that the ARIMA may not be able to capture.

There is a very noticeable difference between the weekday and Saturday data. Try creating a subset of the data for each day of the week or splitting the data into weekday and weekend and applying the model.

If you can obtain temperature data for the same period check if there is a correlation between the temperature and gas usage.

As @Aleksandr Blekh said R does have good packages for ARIMA models

• Do you have any advice related to your first sentence? – marcodena Jul 17 '14 at 23:42
• Create a dataset for each day of the week and fit a model to all seven of them. – germcd Jul 18 '14 at 12:58
• Have you tried a model using temperatures? – germcd Aug 11 '14 at 19:03