# Loss in multi-class classification

I have a multi-class classification task. One of the standard approach in choosing loss function is to use a CrossEntropyLoss. It is a good option when classes are standonlone and not similar to each other.

What if some classes are more similar?

For example, if I have 10 classes, from 0 to 9 and classes with nearby numbers are closer to each other, i.e 4 and 6 are closer to 5 than 0 and 9, etc.

How can I modify CrossEntropyLoss to reflect this fact? Or maybe already exists such loss function?

• What's the problem when some classes are more similar? I think you need to clarify that first. In any case, a weighted cross entropy could be the thing you are searching for. Commented Sep 22, 2021 at 21:25
• @Alessandro because CE can't consider the fact that classes are ordinal. Commented Sep 23, 2021 at 11:22

## 1 Answer

I don't think there is a built-in loss function for what you want - I had the same issue a few years back and I found a custom loss function for this purpose. It is called Ordinal Categorical Classification problem. I have not checked this in a while now but I believe it is still not implemented in Keras.

You can also check this cross-validated question and the references given in the answers.

• Have you encountered ordinal function with two-side ranking? For example, ordinal classes are [0, 1, 2, 3, 4] and $P_{arg\_max} = 2$, then it should be $P(0) < P(1) < P(2) > P(3) > P(4)$. Commented Sep 29, 2021 at 9:56
• No, but I don't really understand it. How is P(3) related to P(1) and P(0)? Commented Sep 29, 2021 at 10:13
• Ranking with property 𝑃(0)<𝑃(1)<𝑃(2) and 𝑃(2)>𝑃(3)>𝑃(4) and their sum (from $P_0$ to $P_4$) equals to zero. Commented Sep 29, 2021 at 10:17