Loss in multi-class classification

I have a multi-class classification task. One of the standard approach in choosing loss function is to use a CrossEntropyLoss. It is a good option when classes are standonlone and not similar to each other.

What if some classes are more similar?

For example, if I have 10 classes, from 0 to 9 and classes with nearby numbers are closer to each other, i.e 4 and 6 are closer to 5 than 0 and 9, etc.

How can I modify CrossEntropyLoss to reflect this fact? Or maybe already exists such loss function?

• What's the problem when some classes are more similar? I think you need to clarify that first. In any case, a weighted cross entropy could be the thing you are searching for. Sep 22 '21 at 21:25
• @Alessandro because CE can't consider the fact that classes are ordinal. Sep 23 '21 at 11:22

• Have you encountered ordinal function with two-side ranking? For example, ordinal classes are [0, 1, 2, 3, 4] and $P_{arg\_max} = 2$, then it should be $P(0) < P(1) < P(2) > P(3) > P(4)$. Sep 29 '21 at 9:56
• Ranking with property 𝑃(0)<𝑃(1)<𝑃(2) and 𝑃(2)>𝑃(3)>𝑃(4) and their sum (from $P_0$ to $P_4$) equals to zero. Sep 29 '21 at 10:17