1
$\begingroup$

I originally trained multiple individual binary classifiers for each label of an image. Then, I realized I can train a single multilabel model for this task. I used binary_cross_entropy loss for this instead of categorical_cross_entropy, but besides for changing the loss function I made no major changes. However, I find that the multilabel classifier still substantially underperforms the individual label classifiers. Is this common and to be expected? Are there any tricks I am missing?

Thanks!

$\endgroup$
1
  • $\begingroup$ Are you sure that you mean multilabel and not multiclass? Because multilabel classification is equivalent to multiple individual binary lassifiers, whereas multiclass is a single model taking all the classes into account. $\endgroup$
    – Erwan
    Mar 12, 2022 at 16:14

1 Answer 1

1
$\begingroup$

It is expected for at least two reasons:

  1. Extra supervision:

The multilabel classifier can use the extra information about other classes eg. by inferring "I want to find horses. I also know houses and dogs. While horses/houses are easy to divide, I have to be careful about the division of horses and dogs.". More concrete: Think of it as transfer learning. Eg. when using any kind of neural network, early feature-extraction layers profit from the extra loss and are learned more generally when you train a single model compared to many binary models.

  1. Class balance

Classes in your multilabel problem are guaranteed to be more balanced than classes in your binary problems are.

$\endgroup$
1
  • $\begingroup$ ps.: ofc. O mean multiclass classifier and not multilabel classifier, as I am sure the questioner did. $\endgroup$
    – JDornheim
    Mar 13, 2022 at 11:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.