I am reading the excellent Hands-on Machine Learning with Scikit-Learn and TensorFlow and in chapter 10, the author says:

"For the output layer, the softmax activation function is generally a good choice for classification tasks (when the classes are mutually exclusive). For regression tasks, you can simply use no activation function at all."

All classification problems I can think of (binary classification, image classification, etc) generate mutually exclusive classes.

Can someone give me a few examples of non-mutual exclusive classification problems?

  • 2
    $\begingroup$ Think of the task of properly tagging an untagged Stack Overflow question. You can always have some overlap area between any two tags, there's no mutual exclusion. $\endgroup$
    – Mephy
    Commented Jul 13, 2017 at 13:59

2 Answers 2


For example, due to the complexity of the images in the ImageNet database. Algorithms will often use hundreds or thousands of output nodes to be capable of classifying a large array of different things. Researchers also relax the cost function and allow the $k$ highest outputs to be considered. If one of these is correct then the example is considered to have been correctly classified. Furthermore with the existence of such complex data, often times a certain object might be a subset of another object.

For example, consider the picture of a human being. If the algorithm detects a human with the highest activation at the output node, but also has high activations for hands, eyes, face. This is not wrong! The image does in fact contain these objects as well and them being detected by the model is encouraging.


Classification with non-mutually-exclusive classes is known as multi-label classification.

This site has a tag for such problems: .

And wikipedia: https://en.wikipedia.org/wiki/Multi-label_classification


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.