7
$\begingroup$

The customary objective function for multi-label (e.g. M labels) classification is binary cross-entropy. The problem is, if we use binary cross-entropy, we are assuming that the output labels are independent of each other, turning the problem to M independent binary classification problems. Is there any suitable objective function that makes the output labels to be dependent on each other?

Improve this question
$\endgroup$
4
  • $\begingroup$ I guess the main idea about the whole concept is that the random process are i.i.d. $\endgroup$
    – Green Falcon
    May 30, 2018 at 20:03
  • 1
    $\begingroup$ That objective would be an interesting objective to try, I am not aware of any! At the same time, how would you think one can take advantage of? What is your domain (target) that you are sure these labels are dependent? Take CelebA multilabel dataset (mmlab.ie.cuhk.edu.hk/projects/CelebA.html), some labels could be considered dependent Female/Lipstick/etc., but there are others are not, in which one could benefit from being them to be independent. $\endgroup$
    – TwinPenguins
    Aug 12, 2018 at 18:31
  • 1
    $\begingroup$ @MajidMortazavi Thanks for time and consideration. The application is medical coding. The advantage is that different diseases have something in common. Thanks for introducing a new dataset to me. $\endgroup$
    – pythinker
    Aug 12, 2018 at 18:45
  • 1
    $\begingroup$ If it is a neural network then all the classification is done together till the last layer where they are separated by using binary cross entropy. My understanding is that in neural networks like for image recognition the recognition is connected on all previous layers except the last one. So for neural networks it is not a big deal that the last layer predicts separately every category. $\endgroup$
    – keiv.fly
    Oct 13, 2018 at 22:41

1 Answer 1

Reset to default
1
$\begingroup$

What you're looking for is called "cost-sensitive classification". Most methods however don't work with label similarities, but rather with relative penalties for different types of misclassifications.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.