I have a multi-class classification task. One of the standard approach in choosing loss function is to use a CrossEntropyLoss. It is a good option when classes are standonlone and not similar to each other.

What if some classes are more similar?

For example, if I have 10 classes, from 0 to 9 and classes with nearby numbers are closer to each other, i.e 4 and 6 are closer to 5 than 0 and 9, etc.

How can I modify CrossEntropyLoss to reflect this fact? Or maybe already exists such loss function?

  • $\begingroup$ What's the problem when some classes are more similar? I think you need to clarify that first. In any case, a weighted cross entropy could be the thing you are searching for. $\endgroup$
    – Alessandro
    Commented Sep 22, 2021 at 21:25
  • $\begingroup$ @Alessandro because CE can't consider the fact that classes are ordinal. $\endgroup$ Commented Sep 23, 2021 at 11:22

1 Answer 1


I don't think there is a built-in loss function for what you want - I had the same issue a few years back and I found a custom loss function for this purpose. It is called Ordinal Categorical Classification problem. I have not checked this in a while now but I believe it is still not implemented in Keras.

You can also check this cross-validated question and the references given in the answers.

  • $\begingroup$ Have you encountered ordinal function with two-side ranking? For example, ordinal classes are [0, 1, 2, 3, 4] and $P_{arg\_max} = 2$, then it should be $P(0) < P(1) < P(2) > P(3) > P(4)$. $\endgroup$ Commented Sep 29, 2021 at 9:56
  • $\begingroup$ No, but I don't really understand it. How is P(3) related to P(1) and P(0)? $\endgroup$
    – serali
    Commented Sep 29, 2021 at 10:13
  • $\begingroup$ Ranking with property ๐‘ƒ(0)<๐‘ƒ(1)<๐‘ƒ(2) and ๐‘ƒ(2)>๐‘ƒ(3)>๐‘ƒ(4) and their sum (from $P_0$ to $P_4$) equals to zero. $\endgroup$ Commented Sep 29, 2021 at 10:17

Your Answer

By clicking โ€œPost Your Answerโ€, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.