# Possible to correct an actual cell state in LSTM via gradient?

Why in LSTM we calculate gradient w.r.t weights, but not w.r.t the cell state?

Is it theoretically possible to correct the contents of the cell state, and what would it result in?

I understand that weights are like a "set of skills", so that network can respond correctly to the input, even gazillions of iterations later. The cell-state is an understanding of what's going on in the past, up to the start of the current minibatch.

So why not to correct the value stored in the cell state? It would be very useful if we carry the cell state forward, between minibatches.

https://stackoverflow.com/a/44183738/9007125]

Generally it is recommended to reset state after each epoch, as the state may grow for too long and become unstable. However in my experience with small size datasets (20,000- 40,000 samples) resetting or not resetting the state after an epoch does not make much of a difference to the end result. For bigger datasets it may make a difference.

Is it theoretically possible to correct the contents of the cell state, and what would it result in?

Yes it is. Using back propagation, it is possible to get the gradient of any value that affects a well-defined output. For training data, that includes all current cell outputs - in fact these are necessary to calculate as an interim step in order to get the gradients of the weights. Once you have the gradients of a cost or error function, then you can perform a step of gradient descent in order to discover a value that would result in a lower error for given training data.

In usual training scenarios you do not alter neuron outputs after they have been calculated using feed-forward, because these are not parameters of the model. Typical reasons to alter neuron values (or even the input) are in order to view what the ideal state might be in a given scenario. If your state can be visualised through some decoder - maybe even another neural network - then this could allow you to see the difference between actual internal state, and a potentially better one. That could be useful in error analysis for example.

So why not to correct the value stored in the cell state?

That's because in usual training scenarios, you are creating a network that predicts an output value. You can calculate the right corrections for training data, but not when predicting, because the whole point of predicting is to estimate a label that you do not already have. As such, you want to alter your function parameters, and not interim values.

It would be very useful if we carry the cell state forward, between minibatches.

Only during training. In a prediction scenario you usually have no way of calculating the necessary gradients. What you don't want is to train a system that then requires using error values and gradients that you do not have in production.

In some scenarios, such as an online system predicting next item $x_{t+1}$ in a sequence, where you could immediately train based on error after you observed the next item and before you continued the prediction sequence for $x_{t+2}$, you could possibly use the approach. I am not sure whether it would help performance, but in principle it could. If it did help, you'd have to compare the improvement versus other simpler changes such as different hyper-parameters on a network that didn't correct internal state using gradients.

In summary, it is possible your idea would work quite well in an online system with near-immediate feedback. In that case you could think of a set of weights as being "rules to update a belief state from data", and the output of hidden layer neurons as being "a current belief state". When errors occur, it does appear to make sense to update both the rules that led to the error and the current belief that resulted from earlier faulty rules. It is perhaps worth an experiment or two. The main caveat is that the two update processes (for weights and LSTM layer state) would interact and/or adapt to each other, so it may not lead to measurably different performance than just adding more LSTM cells to the layer.

• "you could possibly use the approach. I am not sure whether it would help performance, but in principle it could." So, if we were to train directly in an on-line mode, would it be useful to correct the actual cell state via backprop? What I am thinking of, is to: Perform backprop. Take $C_{t-3}$ and copy it rather than $C_{t}$, to start the next sequence. Perform forwardprop on the next sequence. I would assume copying "earlier cell" would make it more reliable, since it would have a chance to adjust itself with the grad from $C_{t-2}, C_{t-1}, C_{t}$ cells. What do you think? – Kari Feb 3 '18 at 15:57
• so the sequences would overlap by $t-3$ in this case, meaning we have to re-do 3 steps again. But such a starting cell would be much more reliable ...if we were to backprop-correct it – Kari Feb 3 '18 at 16:02
• I am not sure I fully understand your plan. I suggest starting simply with some comparison experiments, before putting lots of time into building the idea further. In many ways, rolling forward and back with immediate corrections might not be that different from just repeating the training once weights have been improved from epoch to epoch . . . or running a mini-batch on a random sample at each step etc. So you want to be comparing with ideas that add a similar amount of complexity and training time. – Neil Slater Feb 3 '18 at 16:57