What's the difference between multi label classification and fuzzy classification?

Is it just the between academics and practitioners in term usage?

Or is theoretical difference of how we consider each sample: as belonging to multiple classes at once or to one fuzzy class?

Or this distinction has some practical meaning of how we build model for classification?

Multi-label classification (Wiki):

Given $$K$$ classes, find a map $$f:X \rightarrow \{0, 1\}^K$$.

Fuzzy classification (a good citation is needed!):

Given $$K$$ classes, find a map $$p: X \rightarrow [0, 1]^K$$ where $$\sum_{k=1}^{K} p(k)=1$$.

In multi-label classification, as defined, there is no "resource limit" on classes compared to fuzzy classification.

For example, a neural network with a softmax layer does fuzzy classification (soft classification). If we only select a class with the highest score, then it will become a single-label classification (hard classification), and if we select top $$k$$ classes, it will be a multi-label classification (again hard classification).

Fuzzy classification:        [0.5, 0.2, 0.3, 0, 0]
Single-label classification: [1,   0,   0,   0, 0]
Multi-label classification:  [1,   0,   1,   0, 0]


As another example for multi-label classification, we could have $$K$$ neural networks for $$K$$ classes with sigmoid outputs, and assign a point to class $$k$$ if output of network $$k$$ is higher than 0.5.

Outputs:                     [0.6, 0.1, 0.6, 0.9, 0.2]
Multi-label classification:  [1,   0,   1,   1,     0]


Practical considerations

As demonstrated in the examples, the key difference is the "resource limit" that exists in fuzzy classification but not in multi-label classification. Including the limit (in the first example), or ignoring it (in the second example) depends on the task. For example, in a classification task that has mutually exclusive labels, we want to include the "resource limit" to impose the "mutually exclusive" assumption on the model.

Note that the $$\sum_{k=1}^{K} p(k)=1$$ restriction in fuzzy classification is merely a "definition", there is no point in arguing about a definition. We can either propose another classification, or argue when to use - and when not to use - such classification.

• Hmm, but I thought that is the point of multi-label classification not to use softmax, because classes don't exclude each other. – DmytroSytro Apr 23 '19 at 14:19
• @DmytroSytro you are right I added another example. – Esmailian Apr 23 '19 at 14:27
• So, does it really matter for fuzzy set that sum of probabilities for all classes equals to 1? – DmytroSytro Apr 23 '19 at 14:29
• @DmytroSytro I've added a section to explain when sum=1 restriction is useful. – Esmailian Apr 23 '19 at 15:31
• Thank you! Still, it kind of confuses me that in fuzzy sets there should be limit on the sum of probabilities, so that sample can't belong to different sets with probability 1 for each set. As I understand it's the result of the truth function in fuzzy logic that can't assign sum of probabilities more than 1. – DmytroSytro Apr 23 '19 at 15:50

A multi label classifier learns to predict class labels using some algorithm and training data. It learns to associate an object's label with some vector containing values for the features. It estimates the probability of a sample belonging to a certain class, based on some condition.

Fuzzy classifiers do the same exact thing, except, it uses fuzzy logic to determine which class a sample belongs to. The data would need to be described using linguistic rules as opposed to the data used by a conventional classifier. When classifying a sample, it would return a "degree of membership" to each class.

• So, fuzzy classifiers aren't machine learning? – DmytroSytro Apr 23 '19 at 13:50
• They definitely are. They are both affected by the mathematical model used to describe the problem. There will be a difference in the design of the model when considering fuzzy vs conventional, but it all stems from the mathematical model nonetheless. The last point where I wrote "degree of membership" can be thought to be synonymous with "probability of a sample being associated with a certain label". – Sterls Apr 23 '19 at 13:57