1
$\begingroup$

I am trying to compare three forecasting techniques:

  • A stationary stochastic Poisson-GEV: where the rate of occurrence of the events is given by a Poisson process and, it's intensity is given by a General extreme value distribution (GEV).

  • An Autoregressive Integrated Moving Average (ARIMA)

  • A Long Short-Term Memory Network (LSTM)

It is possible to compare the Poisson-GEV fit to the data by using the Akaike information criterion (AIC), the best fit would be the lower AIC. Likewise one could compare the LSTM and ARIMA by using root mean square error (RMSE).

However, it has been pretty hard to find a way to get the AIC for the Long Short-Term Memory Network, since the number of parameters is unclear to me. While searching a way around this I found this page. I'm still lost.

Any advice would be great. Thank you very much!

$\endgroup$
0
$\begingroup$

One of the most common ways to compare performance of different models is predictive ability on a hold-out data set.

Slice out data that that models did not see during training. Then compare performance of different models on the same dataset using the same evaluation metric. Root mean square error (RMSE) is an example of an evaluation metric.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.