Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
When we create an RNN in Keras, does it learn an initial hidden layer (ie. $h_0$, like a bias term)? I'm having trouble figuring this out, but I have looked here.
Can anyone highlight a part in the code that shows an initial hidden layer is being learned? Or is it just randomly initializing the hidden layer?
EDIT: The code is here , I'm just having trouble identifying where the hidden layer is initialized and what it is initialized to.
In line 809, you can find that SimpleRNNCell takes a bias_initializer as an argument to the constructor function and it defaults to zeros.
