Why is the softmax activation function used in the output layer for CNNs? Why not just take the highest value of the units in the output layer?
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
If you are only interested in the most likely class, during inference you can skip the softmax. This is fairly common even, and the reason why TensorFlow has specific logit functionality. However, for optimizing your network you need a loss function that makes sense. You have normal labels, what would your loss function be if you only compute the linear combinations?
answered Jul 13, 2018 at 17:14
-
$\begingroup$ OK got it, we need it to train the net, but can omit it when predicting the labels. $\endgroup$ Jul 13, 2018 at 17:16
-
$\begingroup$ Thanks to your answer I found this on stack overflow which explains your suggestion in more depth: stackoverflow.com/questions/41455101/… $\endgroup$ Jul 13, 2018 at 17:28
$\begingroup$
$\endgroup$
Because the softmax layer ensures that the outputs can be interpreted as probabilities: it ensures that every output is between 0 and 1, and the outputs sum to 1. Without the softmax, you can't interpret them that way.
We want to interpret them as probabilities, both so that we can get confidence scores for the output of the classifier (the classifier doesn't just say "dog", it says 96% confidence it is a dog, 1% it is a bird, etc.). More importantly, the standard loss function, the cross-entropy loss, requires that the outputs be interpretable as probabilities. Empirically, using the cross-entropy loss (and a softmax layer) leads to good results for classification tasks -- better than other options.
