I am reading through this article trying to understand the bptt algorithm, in the context of an RNN. However there is one part I don’t understand:
layer_1_delta = (future_layer_1_delta.dot(synapse_h.T) +
layer_2_delta.dot(synapse_1.T)) +
sigmoid_output_to_derivative(layer_1)
# Located on line 99 in the article linked above
I can’t seem to figure out which mathematical formula this links to in the bptt algorithm to understand what it does, so could someone please provide it? Many thanks