# How to calculate prediction error in a LSTM keras

I have an LSTM which I have constructed and run in keras using python. I use this model to predict $$n$$ points into the future for a time series forecasting problem.

When I use a method such as ARIMA to make the forecast I am able to generate prediction errors for my predictions as the model is fit by minimising the MLE using AIC for example.

Is there a way that is currently supported in keras for me to generate prediction errors for my regression predictions? If there isn't, is there a way that I can calculate it myself?

• Do you want confidence intervals? Commented Dec 16, 2018 at 17:54
• I think so, you'll have to forgive me that I am not quite sure on what the difference between a confidence interval and prediction error would be. A lot of the time the terminology seems to be used interchangeably. I would like to be able to say something like the model says in week two the quantity will be $\hat{y}$ and we are 99% sure the actual value $y$ will be in the range $\hat{y}_{\text{lower}}$-$\hat{y}_{\text{upper}}$. Does that make sense? Commented Dec 16, 2018 at 17:58

You may use the technique explained in the article "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning"

Very briefely the technique consists of applying dropout for training and predictions.

In Keras this can be easily applied passing the training argument in the call of the Dropout layer.

import keras

input  = ...
x      = keras.layer.Dense(100)(input)
dp     = keras.layer.Dropout(0.5)(x, training=True)
output = keras.layer.Activation('relu')

model = keral.Model(input, output)


During prediction you get the mean and standard deviations:

T = 1000   # Do 1000 predictions to estimate uncertainty
predictions = np.array([model.predict(X_test)] for _ in range(T)])

pred_mean = results.mean(axis=0)
pre_std   = results.std(axis=0)


You can increase (or descrease) T if you need more (or less) precision.

• A previous answer to a similar question: datascience.stackexchange.com/questions/24403/… Commented Dec 17, 2018 at 1:53
• Thanks for the response, I’ll read through the article and see how it goes. Commented Dec 17, 2018 at 5:18