1
$\begingroup$

I have an LSTM which I have constructed and run in keras using python. I use this model to predict $n$ points into the future for a time series forecasting problem.

When I use a method such as ARIMA to make the forecast I am able to generate prediction errors for my predictions as the model is fit by minimising the MLE using AIC for example.

Is there a way that is currently supported in keras for me to generate prediction errors for my regression predictions? If there isn't, is there a way that I can calculate it myself?

$\endgroup$
2
  • 2
    $\begingroup$ Do you want confidence intervals? $\endgroup$
    – xboard
    Dec 16, 2018 at 17:54
  • 1
    $\begingroup$ I think so, you'll have to forgive me that I am not quite sure on what the difference between a confidence interval and prediction error would be. A lot of the time the terminology seems to be used interchangeably. I would like to be able to say something like the model says in week two the quantity will be $\hat{y}$ and we are 99% sure the actual value $y$ will be in the range $\hat{y}_{\text{lower}}$-$\hat{y}_{\text{upper}}$. Does that make sense? $\endgroup$
    – Aesir
    Dec 16, 2018 at 17:58

1 Answer 1

3
$\begingroup$

You may use the technique explained in the article "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning"

Very briefely the technique consists of applying dropout for training and predictions.

In Keras this can be easily applied passing the training argument in the call of the Dropout layer.

import keras

input  = ...
x      = keras.layer.Dense(100)(input)
dp     = keras.layer.Dropout(0.5)(x, training=True)
output = keras.layer.Activation('relu')

model = keral.Model(input, output)

During prediction you get the mean and standard deviations:

T = 1000   # Do 1000 predictions to estimate uncertainty
predictions = np.array([model.predict(X_test)] for _ in range(T)])

pred_mean = results.mean(axis=0)
pre_std   = results.std(axis=0)

You can increase (or descrease) T if you need more (or less) precision.

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.