Backpropagation Through Time (BPTT) of LSTM

Question

I am currently trying to understand the BPTT for Long Short Term Memory (LSTM) in TensorFlow. I get that the parameter num_steps is used for the range that the RNN is rolled out and the Error backpropagated. I got a general question of how this works.

For reference a repitition of the formulas. I'm referring to: Formulas LSTM (https://arxiv.org/abs/1506.00019)

Question: What paths are backpropagated that many steps? The constant error carousel is created by the formula 5, and the derivate for backpropagation ($s(t)\rightarrow s(t-1)$) is $1$ for all timesteps. This is why LSTMs capture long range dependencies. I get confused with the dependencies of $g(t)$, $i(t)$, $f(t)$ and $o(t)$ of $h(t-1)$. In words: The current gates do not just depend on the input, but also on the last hidden state.

Doesn't this dependency lead to the exploding/vanishing gradients problem again?

If I backpropagate along these connections I get gradients that are not one. Peephole connections essentially lead to the same problem.

Thanks for your help!

Answer 1

Doesn't this dependency lead to the exploding/vanishing gradients problem again?

Absolutely, and you better have vanishing gradients otherwise you have a training problem.

Vanishing gradient in this case is not a bad thing, it is a good thing (unlike in feedforward). Let $C(t)$ be the cost function evaluated at time $t$ and $ W(t)$ be some weight of the network at time $t$. What vanishing gradient in this case means is that $ dC(t)/dW(t-u)$ becomes smaller and smaller as u becomes bigger and bigger. That is good because $ dC(t)/dW = \sum_{u=0}^{num\_steps} dC(t)/dW(t-u) $, so if the gradients didn't vanish in time, then the gradients would explode. So $W$ gets a proper non-vanishing gradient even if the gradients in time vanish because the gradient for $W$ is the sum at all times of the gradient for $W$.

In LSTMs the gradients are sure to vanish in time because the activation functions are sigmoids and tanh's so their derivatives are less than or equal to one, so as they get multiplied they slowly become smaller.

This compares to what is normally called the vanishing gradient problem which occurs when gradients vanish while passing from top layers to bottom layers, because that means that $ dC/dW $ for $W $ of the lower layer is vanishing and so the lower layers don't get trained, only the upper layers get trained.

Also, as mentioned in the comments, the above applies to any RNN, not only LSTMs. What sets LSTMs appart from vanilla RNNs in with regards to this question is the gating functions which allows the LSTM to control what it remembers and what it forgets and how much of the new input it takes in. While the above is true in practice for LSTMs (and is true on average also in theory), in theory, one could have a time step $t$ where the output has ignored the last 10 inputs and only depends on the input 11 timestep back ($t-11$), in which case the gradient for the weights 11 timesteps ago will not have decayed. Of course that means that at the next time step ($t+1$) the gradients for 11 steps ago ($t+1 -11 = t-10$) will be zero because the input was totally disregarded at $t-10$. So on average it averages out and you still have the same situation for LSTMs.

Answer 2

I found a very good explanation for this in the paper "LSTM: A Search Space Odyssey" (https://arxiv.org/abs/1503.04069). Read chapter 3 if you want to understand the history of the trainig algorithms.

There is a version of Truncated BPTT for LSTM which was used first, where the cell state is propagated back many steps, but the gradients along other parts of the LSTM are truncated. In later papers also full-gradient-BPTT ist used, where the gradients along gates and so on are also backpropagated in time.

Hope this helps you guys!

Cheers, Torben