I've created a simple LSTM network for testing

model = tf.keras.Sequential()
model.add(layers.LSTM(32, input_shape = (timesteps, data_dim), recurrent_dropout = 0.2))
model.compile(loss = 'mae', metrics = ['accuracy'], optimizer = tf.train.AdamOptimizer())

I am using dropout method as researched by Yarin Gal here and Lingxue Zhu here and my function for dropout looks like this:

f = K.function([model.layers[0].input, K.learning_phase()],

after training I'm using the function above in the "predict_with_dropout"

def predict_with_dropout(x, f=f, n_iter=100):
    result = np.zeros((n_iter,))
    for iter in range(n_iter):
        result[iter] = f([x, 1])[0]

    return result
results = []

for point in test_X:
    results+= [predict_with_dropout([point])]

results_avg = np.apply_along_axis(np.mean, 1, results)
variance = np.apply_along_axis(np.var, 1, results)

This code is working as expected and as I understand it the "predict_with_dropout" function is using the f-function to re-train the LSTM model 100 times and within those 100 times it is dropping out certain cells of the model.

Is this the correct implementation of the papers or am I missing something? If it is correct - is there any way to speed this up?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.