I have this time series, it's an industry metric, the data is reported daily, and the data spans just over 1 year. Sometimes, the data does not get reported, and for that day, we would just use the previous day's number for today.
I differenced the data and ran the augmented dickey fuller test, and the diff-data is indeed stationary. I tried to use autoregression to make predictions. I computed and plotted the autocorrelation, the lag contributions all seem very low, but nevertheless, i selected the highest lags and did
from statsmodels.tsa.ar_model import AutoReg model = AutoReg(diff_data, lags=lags_list)
where lags_list are the lag indices in a python list, e.g. [1, 2, 5, 9, 19]
I checked to see if the model can give me fair numbers for the diff_data I have, and it gave me very wrong numbers. I don't know if I implemented AutoReg incorrectly, if I missed something or misunderstood something. I am sorta out of tricks now...
Perhaps the diff_data is basically white noise, the lag contributions were all very low
from pandas.plotting import autocorrelation_plot autocorrelation_plot(diff_data) plt.show()
pretty much every lag is inside the 99% confidence band, except for 2 out of 300+ lags, and just barely over the band. Does that mean the diff_data is white noise, if so, what should I do now?
On a separate topic, the prediction is supposed to be used when the real number doesnt come in, instead of copying yesterday's number. I used to think we can do better than this "copyover" method, but now... I'm not so sure anymore.