0
$\begingroup$

I have an LSTM network and I use it to predict the load. I want to get the confidence interval for the prediction. I am not sure that I can get that or not. I have tried and search in in the different platform, however, I could not find the solution. Here is my simple model.

import pandas as pd pd.options.mode.chained_assignment = None # default='warn' import numpy as np

import tensorflow as tf
from datetime import datetime
from tensorflow import keras
from keras.models import Sequential
from keras.layers import LSTM, Dense, Dropout
from keras.layers.recurrent import LSTM
from matplotlib import pyplot as plt
from sklearn.preprocessing import StandardScaler

X_train = np.random.rand(10,5,2)
Y_train = np.random.rand(10,2)
X_test = np.random.rand(3, 5, 2)

model = Sequential()
model.add(LSTM(64, activation='relu', input_shape=(X_train.shape[1], X_train.shape[2]), return_sequences=True))
model.add(LSTM(32, activation='relu', return_sequences=False))
model.add(Dropout(0.2))
model.add(Dense(Y_train.shape[1], kernel_regularizer='l2'))
opt = keras.optimizers.Adam(learning_rate=0.001)
model.compile(optimizer=opt, loss='mse')
history = model.fit(X_train, Y_train, epochs=1, batch_size=200, validation_split=0.1,verbose=1)

prediction = model.predict(X_test) 
$\endgroup$

1 Answer 1

2
$\begingroup$

What you need is tensorflow probability. Indeed, you want to estimate a distribution and over that the interval of confidence for your prediction.

To do so, you cannot use mse loss function, but you need something that somehow compares probability distributions. One possibility is the likelihood function (better, as you want a loss to minimise the negative (log)likelihood).

Try to modify your model as follows:

import tensorflow as tf
import tensorflow_probability as tfp
tfd = tfp.distributions

model = Sequential()
model.add(LSTM(64, activation='relu', input_shape=(X_train.shape[1], X_train.shape[2]), return_sequences=True))
model.add(LSTM(32, activation='relu', return_sequences=False))
model.add(Dropout(0.2))
model.add(Dense(2, activation='relu')) # 2 as you want to predict mean and variance
model.add(tfp.layers.DistributionLambda(
    lambda t: tfd.Normal(loc=t[...,0], 
                         scale=0.01*tf.math.softplus(t[...,1])),
    name='normal_dist')) # note this

negloglik = lambda y, p_y: -p_y.log_prob(y) # note this
opt = keras.optimizers.Adam(learning_rate=0.001)
model.compile(optimizer=opt, loss=negloglik)
history = model.fit(train_dataset, epochs=1, batch_size=200, validation_split=0.1,verbose=1)

The only thing you have to carefully check is that since you are using a custom loss function, you need to pass tensors to your model, hence you have to convert your numpy arrays into tf.dataset

train_dataset = tf.data.Dataset.from_tensor_slices((X_train, Y_train))
test_dataset = tf.data.Dataset.from_tensor_slices((X_test, Y_test))

Finally, to get the Confidence Interval over predictions

mean = lambda x: x.mean().numpy().flatten() # multiply by the scaling factor if you used a scaler
sd = lambda x: x.stddev().numpy().flatten() 


def conf_int(pred):
    """95% confidence interval"""
    return np.array([mean(pred) - 2*sd(pred), mean(pred) + 2*sd(pred)]
$\endgroup$
3
  • $\begingroup$ I use 2 dense for output as you specifiy call but get only one output as values!? $\endgroup$ Jul 4, 2023 at 21:19
  • $\begingroup$ @JonathanRoy I do not understand where do you get one output $\endgroup$
    – Oscar
    Jul 6, 2023 at 6:14
  • 1
    $\begingroup$ I find my error, I feed my model with tf.dataset the predict for probabilist was not the good type (tensorflow_probability.python.layers.internal.distribution_tensor_coercible._TensorCoercible) $\endgroup$ Jul 6, 2023 at 10:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.