Should estimated probabilities from multi class classification sum to 1

Question

I am using a neural network with sigmoid activation function $h(z) = 1 / {(1+e^{-z})} $ in order to classify image data into 6 categories. When running the trained neural network over new image data, I noticed that the sums of the estimated probability from the hypothesis output for all 6 classes do not always sum to 1. For example given an input image, the hypothesis output for each class might be:

Class 1 --- 0.10

Class 2 --- 0.11

Class 3 --- 0.12

Class 4 --- 0.13

Class 5 --- 0.14

Class 6 --- 0.15

I interpret this image as having a $13%$ probability of being classified into class 6. However, the sum across all classes is < 1.

My intuition says the probability of each class should sum to 1 but again, I am very new to the machine learning world.

Could there be a bug in my code or is this a 'normal' output?

Answer 1

Summing up to one or not both can have their special meaning that I'll try to explain them. If you have classes that are not mutually exclusive, say you have dog and cat classes and they both can exist in an image. In such cases, you should use sigmoid nonlinearity as the output of each class and interpret each one separately. Each one that has a value greater than half can explain the existence of the corresponding label. On the contrary, your inputs may be mutually exclusive which means in each input you may just have a cat or a dog. In this case, you should employ softmax nonlinearity and yes, it sums up to one. The winner would be the one with the highest value.