3
$\begingroup$

What are the downsides of modelling a multi-label problem as a multi-class problem with a single classifier?

Let my clarify what I mean.

There at least two ways that one multi-label problem can be transformed to a multi-class problem with a single classifier (suppose there are N labels at our problem):

1)

Create a class for each element of the powerset of the labels.

Therefore, each element for each combination of the labels.

In this case, the output vector will have $2^N$ length.

2)

Have an output vector of $N$ length (every element of the vector will be one label) but the problem will be treated as a multiclass problem with one classifier.

In this case, let's say the classes which have output probability more than 0.2 will be considered as the classes/labels of this instance/observation.

Obviously, the output probabilities of all the classes should sum up to 1.

What are the implications of transforming a multi-label problem to multi-class problem at each of these cases?

Improve this question
$\endgroup$
4
  • $\begingroup$ Your edit is wrong I'm afraid: the output of the multilabel problem is one member of the powerset, that is a (binary) vector of length N chosen among 2^N possible distinct vectors (btw the empty set is a valid output in a multilabel setting). One could use the representation (1) by allowing only one cell in the vector to be 1 and all the others 0, but that's just an inefficient representation for the same thing as representation (2). Also there's no need to involve probabilities in the output vector itself, and by adding the constraint to sum to 1 one loses the equivalence with multilabel. $\endgroup$
    – Erwan
    Jul 23, 2019 at 10:23
  • $\begingroup$ @Erwan, you are right about the null set, thank you. Regarding the rest of your comment I am not sure what is its purpose. Do you mention reasons that (1) and (2) are not good ideas to transform a multilabel problem to a multiclass problem or you mean that I have written something else wrongly at my post? $\endgroup$
    – Outcast
    Jul 23, 2019 at 11:52
  • $\begingroup$ In the formal definition of a multi-label problem (en.wikipedia.org/wiki/Multi-label_classification), the output is a vector with binary values (one for each label, i.e. size N). Assuming that the goal is to transform a multi-label problem into an equivalent multiclass problem, the output should be identical. In your option 1 I don't see any reason for making the output a 2^N-length vector, it looks like a confusion between the length and the number of possible values. In your option 2 the output is the right length but you add constraints which are not part of the definition ... $\endgroup$
    – Erwan
    Jul 23, 2019 at 12:36
  • $\begingroup$ Let us continue this discussion in chat. $\endgroup$
    – Erwan
    Jul 23, 2019 at 16:46

1 Answer 1

Reset to default
4
$\begingroup$

A multi-label problem is when an instance can have several labels, for instance a system which classifies news articles by topic could do this:

  • instance 1: politics, society
  • instance 2: sports
  • instance 3: culture, sports
  • instance 4: society
  • ...

To turn this into a multiclass problem and still do the exact same task, one needs to create one class for each possible subset existing in the data, for instance:

{ politics-society, sports, culture-sports, society, ...}

If the original multi-labels problem contains $N$ labels, the number of classes in the multiclass problem is $2^N$ in the worst case (number of partitions of the set).

The main problem is that the classifier needs a representative sample of every class in order to perform well. The classes sports and culture-sports (for example) are now independent from the perspective of the classifier, so the class sport cannot benefit anymore from the instances belonging to culture-sports, as it would be the case in the multi-label problem.

So in general one would need many more instances to train a multiclass classifier than a multi-label classifier for doing the same thing.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your answer (upvote). I think that it would be good to tackle also the 2nd case which is described at my edited post above. $\endgroup$
    – Outcast
    Jul 23, 2019 at 9:17
  • $\begingroup$ See comment under the question. The second case you mention is actually the regular way to convert to multiclass, except that probabilities shouldn't be involved in the output vector. $\endgroup$
    – Erwan
    Jul 23, 2019 at 10:27
  • $\begingroup$ Also, regarding your comment here, probabilities are obviously anyways involved in a problem and you just use a threshold to turn them to 1, 0 etc. Probably you know that of course but then I am just wondering what you mean by your comment here :) . $\endgroup$
    – Outcast
    Jul 23, 2019 at 11:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.