Non-mutal exclusive classification task examples

Question

I am reading the excellent Hands-on Machine Learning with Scikit-Learn and TensorFlow and in chapter 10, the author says:

"For the output layer, the softmax activation function is generally a good choice for classification tasks (when the classes are mutually exclusive). For regression tasks, you can simply use no activation function at all."

All classification problems I can think of (binary classification, image classification, etc) generate mutually exclusive classes.

Can someone give me a few examples of non-mutual exclusive classification problems?

Answer 1

For example, due to the complexity of the images in the ImageNet database. Algorithms will often use hundreds or thousands of output nodes to be capable of classifying a large array of different things. Researchers also relax the cost function and allow the $k$ highest outputs to be considered. If one of these is correct then the example is considered to have been correctly classified. Furthermore with the existence of such complex data, often times a certain object might be a subset of another object.

For example, consider the picture of a human being. If the algorithm detects a human with the highest activation at the output node, but also has high activations for hands, eyes, face. This is not wrong! The image does in fact contain these objects as well and them being detected by the model is encouraging.

Answer 2

Classification with non-mutually-exclusive classes is known as multi-label classification.

This site has a tag for such problems: multilabel-classification.

And wikipedia: https://en.wikipedia.org/wiki/Multi-label_classification