Stateful loss function and online regression

Question

Working in tensorflow + keras I'm trying to define a custom loss function. N.B. I'm more interested in the value of the loss rather than the actual value of the predictions (this will be used for anomaly detection).

My network outputs predictions for the next n-steps ahead.

At time t, I receive an input which is the current observation (y_t), I have a history of x time steps into the past. I wish to compare the value y_t, to those predicted for y_t at each of t-1,...t-n.

To do this I envision a 'triangular' array of arrays ranging in length from 1 to n. I add the values predicted to the end their respective arrays and pop the first value from each to produce my y_pred tensor. y_t is then duplicated into a tensor of equal length (y_true) to be used in computing the loss at time t (probably MSE or the like).

The difficulty I am encountering is how to retain the states of these prediction arrays between batches.

The idea is for this to be used in an online fashion where a small update is made to the network at each time-step (batch size 1) without requiring significant delay between receiving an observation and reporting how accurate the model is (as would be required by saving up predictions for the n-steps and calculating after all have been received).

Answer 1

You can frame your problem as reinforcement learning (RL). Reinforcement learning (RL) is sequential decision-making under uncertainty.

"The value of the loss" could be framed as maximizing rewards. The output is the sequence that maximizes reward for n-steps ahead. The "triangular' array of arrays" is the history.

Reinforcement learning (RL) has the ability to make predicts with current best set of parameters, frequently called "exploitation"