So I have the following problem: I realized (while writing my master thesis) that I am still not sure/have vague descriptions of some of the machine learning principles. I already asked one question regard definitions that can be found here.
Now I stumbled over another definition Problem. Here is an excerpt from my thesis (this is in particular about neural-network classification):
If the classes are mutually exclusive (i.e. if a sample $x^{j} = C_{0}$, $x^{j} \neq C_{i}\setminus~C_{0}$ ), the probabilities of all classes add up to one like \begin{equation} \sum_{i} P(x^{j}=C_{i}) = 1. \end{equation} In this case the best practice is to use a softmax activation function for the output neurons. If the classes are not mutually exclusive it would suffice to use a sigmoid output activation function, as the sigmoid function gets independent probabilities for each class \begin{equation} \sum_{i} P(x^{j}=C_{i}) \geq 1. \end{equation}
I already found the following link regarding this topic. However I know that in practise if you don't use softmax activation function in your output layer, the value can be larger than 1 but can a probability be larger 1? Isn't that against its definition?
Is a non-mutual classification really a common case? Can somebody may be linking some cases (paper?) were they needed non-mutual classification?