This is a general question. Can anybody please point me in the direction of how I can compare the forecasts of a 3 level categorical time series by three competing models? I would like to compare the forecasts to the ground truth of what actually occurred. The forecasts are 1 to 6 days ahead. Is it correct to analyse each day ahead separately? We have forecasts for 4 years. Should they be analysed separately to allow for variation in each year or can the years be used more wholistically? The prediction accuracy for each each model for each day is assessed with the Gerrity score. I am an R user.
I am happy to provide more info if required. Thanking you in advance.