1
$\begingroup$

I have an LSTM which I have constructed and run in keras using python. I use this model to predict $n$ points into the future for a time series forecasting problem.

When I use a method such as ARIMA to make the forecast I am able to generate prediction errors for my predictions as the model is fit by minimising the MLE using AIC for example.

Is there a way that is currently supported in keras for me to generate prediction errors for my regression predictions? If there isn't, is there a way that I can calculate it myself?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Do you want confidence intervals? $\endgroup$
    – xboard
    Dec 16, 2018 at 17:54
  • 1
    $\begingroup$ I think so, you'll have to forgive me that I am not quite sure on what the difference between a confidence interval and prediction error would be. A lot of the time the terminology seems to be used interchangeably. I would like to be able to say something like the model says in week two the quantity will be $\hat{y}$ and we are 99% sure the actual value $y$ will be in the range $\hat{y}_{\text{lower}}$-$\hat{y}_{\text{upper}}$. Does that make sense? $\endgroup$
    – Aesir
    Dec 16, 2018 at 17:58

1 Answer 1

Reset to default
3
$\begingroup$

You may use the technique explained in the article "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning"

Very briefely the technique consists of applying dropout for training and predictions.

In Keras this can be easily applied passing the training argument in the call of the Dropout layer.

import keras

input  = ...
x      = keras.layer.Dense(100)(input)
dp     = keras.layer.Dropout(0.5)(x, training=True)
output = keras.layer.Activation('relu')

model = keral.Model(input, output)

During prediction you get the mean and standard deviations:

T = 1000   # Do 1000 predictions to estimate uncertainty
predictions = np.array([model.predict(X_test)] for _ in range(T)])

pred_mean = results.mean(axis=0)
pre_std   = results.std(axis=0)

You can increase (or descrease) T if you need more (or less) precision.

Improve this answer
$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.