When using tf.keras.layers.SimpleRNN,does this SimpleRNN have a hidden state, or does it just use the output value as the hidden state. That is, does it follow the formulas $h_t = \tanh(w_h\cdot h_{t-1} + w_x\cdot x_{t-1}+b_h)$, $y_t = w_o\cdot h_t + b_o$? Or is $h_t=y_t$?

If the input is of length 5, and I make keras.layers.SimpleRNN(1), why are there only 6 parameters according to the summary. There should be 5 for $w_x$, 1 for $b_h$, 1 for $w_h$, and possibly 2 for output, for a total to 7 or 9.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.