2
$\begingroup$

I have a multi-class classification task. One of the standard approach in choosing loss function is to use a CrossEntropyLoss. It is a good option when classes are standonlone and not similar to each other.

What if some classes are more similar?

For example, if I have 10 classes, from 0 to 9 and classes with nearby numbers are closer to each other, i.e 4 and 6 are closer to 5 than 0 and 9, etc.

How can I modify CrossEntropyLoss to reflect this fact? Or maybe already exists such loss function?

Improve this question
$\endgroup$
2
  • $\begingroup$ What's the problem when some classes are more similar? I think you need to clarify that first. In any case, a weighted cross entropy could be the thing you are searching for. $\endgroup$
    – Alessandro
    Commented Sep 22, 2021 at 21:25
  • $\begingroup$ @Alessandro because CE can't consider the fact that classes are ordinal. $\endgroup$
    – Kenenbek Arzymatov
    Commented Sep 23, 2021 at 11:22

1 Answer 1

Reset to default
2
$\begingroup$

I don't think there is a built-in loss function for what you want - I had the same issue a few years back and I found a custom loss function for this purpose. It is called Ordinal Categorical Classification problem. I have not checked this in a while now but I believe it is still not implemented in Keras.

You can also check this cross-validated question and the references given in the answers.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Have you encountered ordinal function with two-side ranking? For example, ordinal classes are [0, 1, 2, 3, 4] and $P_{arg\_max} = 2$, then it should be $P(0) < P(1) < P(2) > P(3) > P(4)$. $\endgroup$
    – Kenenbek Arzymatov
    Commented Sep 29, 2021 at 9:56
  • $\begingroup$ No, but I don't really understand it. How is P(3) related to P(1) and P(0)? $\endgroup$
    – serali
    Commented Sep 29, 2021 at 10:13
  • $\begingroup$ Ranking with property 𝑃(0)<𝑃(1)<𝑃(2) and 𝑃(2)>𝑃(3)>𝑃(4) and their sum (from $P_0$ to $P_4$) equals to zero. $\endgroup$
    – Kenenbek Arzymatov
    Commented Sep 29, 2021 at 10:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.