Please note that you should ask one question at a time, but as your questions are connected to each other, it is ok.
I've noticed most librairies don't have options to set a validation larger than the forecast horizon, is there a reason?
There might be confusion between forecast horizon and validation: both should be connected actually.
The validation is larger to see if the model is working well or not predicting 3 days ahead. For instance, if you train your model on 600 days predicting 3 days, it will be trained each day (using also the N previous ones) to predict the next 3 ones. Then, the validation is on 30 days and it will apply the same 3 days horizon prediction every day to see whether the predictions are valid or not.
365 days is not a multiple of my forecast horizon (3 days). Is it a problem?
Not at all: the more data you have to train, the better it is, but you have to define the forecast horizon.
Here is a code example to predict 3 days with ARIMA in Python:
from pandas import read_csv
from statsmodels.tsa.arima.model import ARIMA
# create a differenced series
def difference(dataset, interval=1):
diff = list()
for i in range(interval, len(dataset)):
value = dataset[i] - dataset[i - interval]
# invert differenced value
def inverse_difference(history, yhat, interval=1):
return yhat + history[-interval]
# load dataset
series = read_csv('dataset.csv', header=0)
# seasonal difference
X = series.values
days_in_year = 365
differenced = difference(X, days_in_year)
# fit model
model = ARIMA(differenced, order=(7,0,1))
model_fit = model.fit()
# multi-step out-of-sample forecast
forecast = model_fit.forecast(steps=3)
# invert the differenced forecast to something usable
history = [x for x in X]
day = 1
for yhat in forecast:
inverted = inverse_difference(history, yhat, days_in_year)
print('Day %d: %f' % (day, inverted))
day += 1
The librairies I'm using compute the RMSE for one horizon. If a do the RMSE average of on all horizons, I don't get the RMSE of the full year since RMSE is not a linear metric. Should I stop using the RMSE computed by the libraries?
RMSE is a good option and it could be used together with the Aikake Information Criteria to evaluate the model quality.
How to build ARIMA model in Python
Akaike Information Criterion
When evaluating my models, should I consider every day as a forecast day, or only forecasting every forecast horizon (i.e. every 3 days here)
Generally speaking, the more days you try to predict, the less good result you can get because models learns also noise which is amplified day after day.
That's why, it is recommendable to start with 1 day forecast, see if the results are satisfactory, then increase the forecast horizon by one and repeat the process until the results are not interesting anymore.
It could be also interesting to compare with other models (Random Forest, LSTM, Prophet, etc.) and use them in different horizon forecast.