Algorithm for backpropagation through time

Question

I am reading through this article trying to understand the bptt algorithm, in the context of an RNN. However there is one part I don’t understand:

layer_1_delta = (future_layer_1_delta.dot(synapse_h.T) + 
    layer_2_delta.dot(synapse_1.T)) +     
    sigmoid_output_to_derivative(layer_1)

# Located on line 99 in the article linked above 

I can’t seem to figure out which mathematical formula this links to in the bptt algorithm to understand what it does, so could someone please provide it? Many thanks

Answer 1

As far as I can make out, this is simply the error aggregated at layer_1, meaning it is the sum of the errors from the current hidden layer, plus the error from all future hidden layers (in this case, that is only one layer: layer_2) plus the error from the output (prediction) layer - sigmoid_out.

I think that line of code (#99) is performing the loss update with respect to a chosen layer and all layers that are ahead in time - so closer to the network's output layer, but as we are moving backwards during backpropagation. It corresponds then, I believe, to equation 3.24 and 3.25 in Ilya Sutskever's PhD Thesis (on page 35).

So you are summing up the gradients of all layers ahead of the current layer, as well as the gradient of the output layer (the sigmoid). Equation 3.25 from the link above looks like this:

$$ \frac{\delta L}{\delta W} = \sum_{t=1}^{T-1} \bigg( W'^T (r_{t+1} \odot (1 - r_{t+1}) \odot \frac{\delta L}{\delta r_{t+1}}) \bigg) v_t^T + \sum_{t=1}^{T-1} \frac{\delta log \hat P (v_t | r_{t-1})}{\delta W} $$

... where , where $v_t$ are the input variables and $r_t$ are the RNN’s hidden variables (all of which are deterministic). (from page 34)

The explanation given in the thesis:

The first summation in eq. 3.25 corresponds to the use of $W$ for computing $r_t$, and the second summation arises from the use of $W$ as RBM parameters for $log \hat P(v_t |r_{t−1})$, so each $∂log \hat P(v_{t+1}|r_t)/∂W$ term is computed with CD (contrastive divergence). Computing $∂L/∂rt$ is done most conveniently with a single backward pass through the sequence. It is also seen that the gradient of the RTRBM ( Recurrent Temporal Restricted Boltzmann Machine) would be computed exactly if CD were replaced with the derivatives of the RBM’s log probability.

bold text added by me

The main leap here to relate to your example blog is that the it equates the log probabilities of a Restricted Boltzmann Machine to the loss of a generative RNN. If you are interested in more details, I would recommend reading either all of Chapter 3, or perhaps just sections 3.9 and 3.10.

---

I posted that equation because I could see a version that so closely matches the code in that blog in either of the original papers that propose backpropagation through time. Additionally, none seem to be free to read. You can find them link on the relevant Wikipedia page.

Have a look at this related paper, written by one of the original authors of BPTT. Backpropagation Through Time: What It Does and How to Do It.