# Recurrent Neural Network (RNN) Vanishing gradient problem - Why does it affect earlier timesteps more?

I understand the concept of backpropagation in standard neural networks and backpropagation through time with RNNs, why this causes exponentially smaller gradients at earlier time steps and most of the maths behind it all, but what I don’t understand is why this affects the earlier timestep in particular? Since, the parameters (weights) in the RNN are all shared between timesteps, why is it that the earlier timestep is more affected? Wouldn’t they all be affected since they all share the same badly optimized weight which is never updated due to the many small terms in the multiplicative product which produces $$\frac{\partial E}{\partial w}$$? I feel like I'm totally misunderstanding something here. Many thanks

• Thank you very much. Just to confirm my understanding, by contribution here, you mean the contribution to the sum which computes the partial derivate $\frac{\partial E}{\partial w}$? And this sum is formed based on the product rule right? Apr 22 at 8:47