# Measuring uncertainty in an LSTM network using dropout in keras/tensorflow

I've created a simple LSTM network for testing

model = tf.keras.Sequential()
model.add(layers.LSTM(32, input_shape = (timesteps, data_dim), recurrent_dropout = 0.2))
model.compile(loss = 'mae', metrics = ['accuracy'], optimizer = tf.train.AdamOptimizer())


I am using dropout method as researched by Yarin Gal here and Lingxue Zhu here and my function for dropout looks like this:

f = K.function([model.layers[0].input, K.learning_phase()],
[model.layers[-1].output])


after training I'm using the function above in the "predict_with_dropout"

def predict_with_dropout(x, f=f, n_iter=100):
result = np.zeros((n_iter,))
#print(f([x,1]))
for iter in range(n_iter):
result[iter] = f([x, 1])[0]

return result
results = []

for point in test_X:
results+= [predict_with_dropout([point])]

results_avg = np.apply_along_axis(np.mean, 1, results)
variance = np.apply_along_axis(np.var, 1, results)


This code is working as expected and as I understand it the "predict_with_dropout" function is using the f-function to re-train the LSTM model 100 times and within those 100 times it is dropping out certain cells of the model.

Is this the correct implementation of the papers or am I missing something? If it is correct - is there any way to speed this up?