7
$\begingroup$

The customary objective function for multi-label (e.g. M labels) classification is binary cross-entropy. The problem is, if we use binary cross-entropy, we are assuming that the output labels are independent of each other, turning the problem to M independent binary classification problems. Is there any suitable objective function that makes the output labels to be dependent on each other?

$\endgroup$
  • $\begingroup$ I guess the main idea about the whole concept is that the random process are i.i.d. $\endgroup$ – Media May 30 '18 at 20:03
  • 1
    $\begingroup$ That objective would be an interesting objective to try, I am not aware of any! At the same time, how would you think one can take advantage of? What is your domain (target) that you are sure these labels are dependent? Take CelebA multilabel dataset (mmlab.ie.cuhk.edu.hk/projects/CelebA.html), some labels could be considered dependent Female/Lipstick/etc., but there are others are not, in which one could benefit from being them to be independent. $\endgroup$ – TwinPenguins Aug 12 '18 at 18:31
  • 1
    $\begingroup$ @MajidMortazavi Thanks for time and consideration. The application is medical coding. The advantage is that different diseases have something in common. Thanks for introducing a new dataset to me. $\endgroup$ – pythinker Aug 12 '18 at 18:45
  • 1
    $\begingroup$ If it is a neural network then all the classification is done together till the last layer where they are separated by using binary cross entropy. My understanding is that in neural networks like for image recognition the recognition is connected on all previous layers except the last one. So for neural networks it is not a big deal that the last layer predicts separately every category. $\endgroup$ – keiv.fly Oct 13 '18 at 22:41
1
$\begingroup$

What you're looking for is called "cost-sensitive classification". Most methods however don't work with label similarities, but rather with relative penalties for different types of misclassifications.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.