1
$\begingroup$

When we create an RNN in Keras, does it learn an initial hidden layer (ie. $h_0$, like a bias term)? I'm having trouble figuring this out, but I have looked here.

Can anyone highlight a part in the code that shows an initial hidden layer is being learned? Or is it just randomly initializing the hidden layer?

EDIT: The code is here , I'm just having trouble identifying where the hidden layer is initialized and what it is initialized to.

$\endgroup$
0
$\begingroup$

In line 809, you can find that SimpleRNNCell takes a bias_initializer as an argument to the constructor function and it defaults to zeros.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.