I am building a program to implement a solution for multilabel classification. It's an interesting problem because most of the examples (~90%) actually only have one class, but some have multiple classes.

Additionally there are certain classes which we know to be mutually exclusive, i.e. if an example is in class A then it cannot be in class B, and vice-versa.

The way it's currently implemented is to identify all the classes which are mutually exclusive and place them within their own classifier, then build a separate classifier for each of the others:

Classifier 1: Predicts A (or OTHER)
Classifier 2: Predicts B (or OTHER)
Classifier 3: Predicts C,D,E, or F (or OTHER)

When the classifier runs, it gets predictions from all 3 classifiers and then combines them. So if classifier 1 predicts A, and classifier 2 and 3 both predict OTHER, then the result would be A. If instead classifier 2 had predicted B, then the result would be A,B.

So my questions are:

  1. Does this seem like a reasonable implementation to you?
  2. It seems like the downside is that we could end up with A and B too many times when really we wanted them to be exclusive more often. In that case we could try to improve things with a classifier that controls the degree of exclusivity by predicting A, B, A-and-B instead. But is that likely to represent an improvement, or would it be unnecessary?

In an ideal world if the A and B classifiers are functioning correctly I think it would be unnecessary, but I don't know if combining the decision into a single classifier would allow the classifier to learn the boundaries "better" in some way.

  • $\begingroup$ How complicated are the mutual exclusivity rules? This might determine what approach you can take. $\endgroup$ – Imran Feb 16 '18 at 19:17
  • $\begingroup$ Also what does your cost look like? Is it all-or-nothing? In that case you might want to pick A or B if both are predicted, whereas in some other cases it might be best to predict A and B even if you know that's not possible. $\endgroup$ – Imran Feb 16 '18 at 19:21
  • $\begingroup$ The exclusivity roles are quite simple, it’s just that we know for certain a couple of classes can never go together, and then I think we’ve observed empirically that other classes are never combined. This is currently encoded in the scheme you see where all those classes are combined into the third classifier which will only predict one of them $\endgroup$ – Stephen Feb 16 '18 at 23:25
  • $\begingroup$ I’m not sure what you mean by “all or nothing cost” though $\endgroup$ – Stephen Feb 16 '18 at 23:25
  • $\begingroup$ I mean can you get partial credit on a training example if some classes are labeled correctly but others are not? $\endgroup$ – Imran Feb 16 '18 at 23:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.