I've created a simple LSTM network for testing
model = tf.keras.Sequential() model.add(layers.LSTM(32, input_shape = (timesteps, data_dim), recurrent_dropout = 0.2)) model.add(layers.Dense(1)) model.compile(loss = 'mae', metrics = ['accuracy'], optimizer = tf.train.AdamOptimizer())
f = K.function([model.layers.input, K.learning_phase()], [model.layers[-1].output])
after training I'm using the function above in the "predict_with_dropout"
def predict_with_dropout(x, f=f, n_iter=100): result = np.zeros((n_iter,)) #print(f([x,1])) for iter in range(n_iter): result[iter] = f([x, 1]) return result results =  for point in test_X: results+= [predict_with_dropout([point])] results_avg = np.apply_along_axis(np.mean, 1, results) variance = np.apply_along_axis(np.var, 1, results)
This code is working as expected and as I understand it the "predict_with_dropout" function is using the f-function to re-train the LSTM model 100 times and within those 100 times it is dropping out certain cells of the model.
Is this the correct implementation of the papers or am I missing something? If it is correct - is there any way to speed this up?