Starting from Yarin Gal's research paper on using Dropout as a Bayesian Approximation (https://arxiv.org/pdf/1506.02142.pdf), I am trying to apply this concept to my Sequence Prediction model. My model comprises of 2 LSTM layers, followed by a relu dense layer, followed by a softmax layer. A dropout layer is added after each of the layers.
I've found some implementations for measuring the uncertainty of a deep neural network (like this one here: https://fairyonice.github.io/Measure-the-uncertainty-in-deep-learning-models-using-dropout.html), but all of them seem to be applicable to dense layers rather than LSTM layers.
So in summary, what I am asking about is:
- How can I turn on Dropout during testing for LSTM layers?
- How can I measure my model's uncertainty?