In multiclass classification, the assumption is that every instance has exactly one class. Example: a poll asks people their favourite colour among blue (B), yellow (Y) or red (R). Each instance represents a person's answer, either B, Y or R. The "one vs. rest" method means that 3 binary classifiers are trained:
- "B" vs "not B", where the Y and R instances are labelled "not B"
- "Y" vs "not Y", where the B and R instances are labelled "not Y"
- "R" vs "not R", where the B and Y instances are labelled "not R"
These models are not independent by assumption, for example:
- if the class is B then it cannot be Y or R.
- if the class is not Y then it's either B or R.
In probabilistic terms this translates as a distribution which sums to 1, because if a class has a high probability then it's impossible that any other class also has a high probability (complement). This is why the probabilities predicted by the binary classifiers are each divided by the sum (see Ben's answer for details).
Note: by contrast multi-label classification allows every instance to have any number of classes. In the example above it's as if the poll asks people to say whether they like each colour B, Y, R. A person might like all 3 colours or none of them. This implies that the binary classifiers are independent:
- For "B vs not B", both the B and "not B" classes can contain instances which also have Y or R (or both).
- As a consequence the classifiers are independent: knowing that an instance has class B doesn't imply anything about the other classes.