Working in tensorflow + keras I'm trying to define a custom loss function. N.B. I'm more interested in the value of the loss rather than the actual value of the predictions (this will be used for anomaly detection).

My network outputs predictions for the next n-steps ahead.

At time t, I receive an input which is the current observation (y_t), I have a history of x time steps into the past. I wish to compare the value y_t, to those predicted for y_t at each of t-1,...t-n.

To do this I envision a 'triangular' array of arrays ranging in length from 1 to n. I add the values predicted to the end their respective arrays and pop the first value from each to produce my y_pred tensor. y_t is then duplicated into a tensor of equal length (y_true) to be used in computing the loss at time t (probably MSE or the like).

The difficulty I am encountering is how to retain the states of these prediction arrays between batches.

The idea is for this to be used in an online fashion where a small update is made to the network at each time-step (batch size 1) without requiring significant delay between receiving an observation and reporting how accurate the model is (as would be required by saving up predictions for the n-steps and calculating after all have been received).

  • $\begingroup$ I'm also having a similar issue, and if you've managed to figure it out, I would appreciate it if you shared your solution. $\endgroup$ Apr 14, 2020 at 23:22

1 Answer 1


You can frame your problem as reinforcement learning (RL). Reinforcement learning (RL) is sequential decision-making under uncertainty.

"The value of the loss" could be framed as maximizing rewards. The output is the sequence that maximizes reward for n-steps ahead. The "triangular' array of arrays" is the history.

Reinforcement learning (RL) has the ability to make predicts with current best set of parameters, frequently called "exploitation"


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.