I have a multi-class classification task. One of the standard approach in choosing loss function is to use a CrossEntropyLoss. It is a good option when classes are standonlone and not similar to each other.

What if some classes are more similar?

For example, if I have 10 classes, from 0 to 9 and classes with nearby numbers are closer to each other, i.e 4 and 6 are closer to 5 than 0 and 9, etc.

How can I modify CrossEntropyLoss to reflect this fact? Or maybe already exists such loss function?

  • $\begingroup$ What's the problem when some classes are more similar? I think you need to clarify that first. In any case, a weighted cross entropy could be the thing you are searching for. $\endgroup$
    – Alessandro
    Sep 22, 2021 at 21:25
  • $\begingroup$ @Alessandro because CE can't consider the fact that classes are ordinal. $\endgroup$ Sep 23, 2021 at 11:22

1 Answer 1


I don't think there is a built-in loss function for what you want - I had the same issue a few years back and I found a custom loss function for this purpose. It is called Ordinal Categorical Classification problem. I have not checked this in a while now but I believe it is still not implemented in Keras.

You can also check this cross-validated question and the references given in the answers.

  • $\begingroup$ Have you encountered ordinal function with two-side ranking? For example, ordinal classes are [0, 1, 2, 3, 4] and $P_{arg\_max} = 2$, then it should be $P(0) < P(1) < P(2) > P(3) > P(4)$. $\endgroup$ Sep 29, 2021 at 9:56
  • $\begingroup$ No, but I don't really understand it. How is P(3) related to P(1) and P(0)? $\endgroup$
    – serali
    Sep 29, 2021 at 10:13
  • $\begingroup$ Ranking with property 𝑃(0)<𝑃(1)<𝑃(2) and 𝑃(2)>𝑃(3)>𝑃(4) and their sum (from $P_0$ to $P_4$) equals to zero. $\endgroup$ Sep 29, 2021 at 10:17

Your Answer

By clicking β€œPost Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.