I am learning about backpropagation in LSTM. I have been studying an article and watching two videos on the topic. The videos 1 and 2 repeat all the information from the article, but with additional comments. The backpropagation formulas for vanilla RNNs are correct, as I have checked them and compared them with other users' formulas. However, the formulas for LSTMs seem strange to me. LSTM architectures are more complex than RNN architectures, so I expect the backpropagation formulas to be more complex as well. In the videos and article, however, the gradient for weights does not even have double summations, compared to double summations plus the product operator in RNNs. In my opinion, these formulas are incorrect. The authors seem to have ignored the fact that ht-1 also depends on Who, Whi, Whf, Whc etc...dWhi =



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.