Gradient Checking LSTM - how to get change in Cost across timesteps?

Question

I am performing gradient check for my LSTM which has 4 timesteps. The LSTM looks as follows:

   01       01       01       01
   ^        ^        ^        ^
  LSTM --> LSTM --> LSTM --> LSTM
   ^        ^        ^        ^
   11       11       11       11

So, at every timestep we are feeding in vector {1,1} and expect {0,1} at the output.

Assume I perturb the weight inside LSTM, then perform 4 forward props - one for each timestep - how do I now get delta of the cost function that this single perturbation has caused?

Am I allowed to simply add-up the change in Cost from all 4 timesteps to treat it as derivative estimate?

---

Also, should I perform it as follows for LSTM:

1. perturb a single weight upwards
2. forward prop 4 timesteps
3. perturb the weight downwards
4. forward prop 4 timesteps
5. get 4 deltas
6. sum the 4 deltas to get a total change in Cost

or

1. Set N=0
2. perturb the weight upwards
3. foward prop at a particular timestep N
4. perturb the weight downwards
5. forward prop at a particular timestep N
6. get single delta, store it away
7. increment N
8. until N not equal 4 return to step 2)
9. sum the 4 deltas to get a total change in Cost

The second approach somehow seems more correct, because LSTM will have a hidden state ..Is this correct intuition or it won't matter?

Answer 1

Interesting question.

Like standard backpropagation, [backpropagation through time] consists of a repeated application of the chain rule. The subtlety is that, for recurrent networks, the loss function depends on the activation of the hidden layer not only through its influence on the output layer, but also through its influence on the hidden layer at the next timestep.

It looks like both approaches would have similar results but at a different granularity (adding noise at different levels) this is because the backprop is not really disturbed by this addition as it is still chained from last time step to the first. Hence it boils down to why you actually want to add noise in the first place, as mentioned in this paper , sections IIIC and IV.

Answer 2

Answering my own question several months later (after reading the answer by @SanjayKrishna).

My 'approach 1' seems more correct, but causes more hustle than is actually needed.

Don't forget that the Cost is the Mean Squared Error. In my specific case it's the average of the errors from each timestep. It's this MSE which allows us to see the "delta".

$$C = MSE= \frac{1}{T}\sum_{t=0}^{t=T}(actual_t-wanted_t)^2$$

Thus, we should do the following:

1. peturb a single weight upwards
2. perform a full forward prop (for example, 15 timesteps), get the cost_a from your MSE. It should be just a single scalar value.
3. peturb the weight downwards
4. redo the full fwd prop to obtain the cost_b from your MSE which is another scalar value.
5. compute the delta (just a single scalar value) by subtracting cost_a from cost_b
6. compare the delta to your gradient for that was computed during BackPropThroughTime (your gradient for that particular weight, estimated from all the timesteps)

Edit

I am actually using something like "Mean SoftMaxedCrossEntropy", not MSE. But the idea is the same: sum up the errors at each timestep, divide by $T$ and that's my Cost.