Although Multi class is different from Multi label classification, whats difference does adding One vs All make in Multi-class.

Edit 1: http://scikit-learn.org/stable/modules/multiclass.html#multilabel-learning

In this Link the part where it mentions supported classifiers under bullets of Multiclass as One-Vs-All and Support Multilabel is confusing.

  • $\begingroup$ Where have you seen the terms? Please add citation for each one. $\endgroup$ Commented Aug 24, 2018 at 12:03

3 Answers 3


In multiclass classification each class is mutually exclusive, but in multilabel classification each class basically represents a different binary classification task.

An example.

Multiclass: Images that could contain a dog, a cat or a frog. Each image contains only one of the animals.


Multilabel: Movie Genre Classification based on poster images. You have a poster image from a movie and want to determine whether the movie is a drama, action, thriller etc. A movie could belong to multiple of the these genres.

So to answer your question, the one-vs-all strategy in multilabel classification basically separates the k binary classification tasks. So using the above example, you would have k binary classifiers, where each one would basically represent each genre. So you would have a binary classifier for drama, one for action, one for thriller etc.


Let's say you have K classes, your multi-label classification model outputs the probability of the input being of each of the K classes:

$$ p(x) = (p_1(x)=p(y=k|x),..., p_K(x)=p(y=K|x)) $$

Another approach is to train K binary classifiers and specializing them in recognizing one class, thus one classifier that recognizes only class k, its output would be a unique probability:

$$ q^k(x) = (q^k_1(x)=p(y=1|x)) $$

The approach is different in:

  • one vs all you train K classifiers, in the multilabel approach you train 1 classifier.

  • you will have K different training datasets as you see the labels for class k the one vs all classifier takes as input data with labels: "k" or "not k".

Your final classification result in the one vs all approach is:

$$ y(x) = argmax_k (q^1_k(x)) $$


In multi-class classification, you can train a classifier to predict the class among the set of possible classes.

However, you could also train one binary classifier for each class, where each classifier is trained to predict whether an elements belongs to the associated class or not. At inference time, you predict with all binary classifiers and then combine the information in a single prediction. This is multi-class one vs all. It allows to turn any binary classification algorithm into a multi-class classification one.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.