I am trying to implement a custom loss function inspired by https://arxiv.org/pdf/2305.10464.pdf. That is:

$ L(\mathbf{x}) = (1-y) \left\lVert \mathbf{x_{true} - \mathbf{x_{pred}}} \right\rVert^2 + y \left\lVert \mathbf{1} - \mathbf{x_{true} - \mathbf{x_{pred}}} \right\rVert^2 $

The idea is that the model should reconstruct normal data as closely as possible while reconstructing anomalous data poorly. Thus, the model aims to put anomalous data outside of the domain description of the normal data. The labels y are purely to help guide the model and not to be predicted (y=1 --> anomaly , y = 0 --> normal ).

Although I find the loss function to be intuitive when comparing just two data points, I feel a bit confused when implementing this in python using keras. My understanding is that the loss function is evaluated per batch (is this really the case?). For the classical MSE loss, this would mean averaging the MSE-loss for all data points within the batch (https://keras.io/api/losses/regression_losses/). With the custom loss function, I am worried that the evaluation of the loss function won't capture the "anomaly-part" of the loss function as this would be lost in the aggregation? I'm unsure how to guarantee that these subtleties are effectively captured during model training. Any insights or suggestions on this matter would be highly valued.


1 Answer 1


worried that the evaluation of the loss function won't capture the "anomaly-part" of the loss function as this would be lost in the aggregation?

You're talking about weighting the minority class, the anomalies. If they have low weight, the model will focus on reconstruction error and may ignore the anomaly class label.

The OP mentions Angiulli et al.'s eqn (2.), with $\lambda$ set to $1$, and $F(\mathbf{x}) = \mathbf{1} - \mathbf{x}$.

Your choice of setting the hyperparameter $\lambda$ to unity suggests you believe there is no strong imbalance of classes in your dataset. But the notion of "anomaly", and the worry you gave voice to, suggest you'd be happier with a larger factor there. In §4.2 the authors recommend using log-spaced factors s.t. $0.1 < \lambda(\alpha) < 2$.

  • $\begingroup$ Thanks for your input! As you suggested, adjusting the weighting of the "anomaly - loss" could address the imbalance effectively. However, hypothetically speaking, if I were to employ sample weights, would it convey similar affects as weighting the "anomaly-loss"? $\endgroup$
    – Gst
    Commented Apr 25 at 21:22
  • $\begingroup$ (A.) I don't deal with hypotheticals, I deal with actual implemented functions, such as the one you kindly offered in OP with λ = 1. Suggest a new parameterization, with accompanying code, if you find this one is unsuitable. (B.) I don't know what distribution of abnormal or anomalous examples you're considering; I've not yet seen that dataset. You have two horses; I've seen neither. It is unclear if you're asking for an original analysis, or that I put words in the mouths of Angiulli, Fassetti, et al. The corresponding author is Angiulli. $\endgroup$
    – J_H
    Commented Apr 25 at 23:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.