1
$\begingroup$

Why is the softmax activation function used in the output layer for CNNs? Why not just take the highest value of the units in the output layer?

$\endgroup$
1
$\begingroup$

If you are only interested in the most likely class, during inference you can skip the softmax. This is fairly common even, and the reason why TensorFlow has specific logit functionality. However, for optimizing your network you need a loss function that makes sense. You have normal labels, what would your loss function be if you only compute the linear combinations?

$\endgroup$
  • $\begingroup$ OK got it, we need it to train the net, but can omit it when predicting the labels. $\endgroup$ – erdal.karaca Jul 13 '18 at 17:16
  • $\begingroup$ Thanks to your answer I found this on stack overflow which explains your suggestion in more depth: stackoverflow.com/questions/41455101/… $\endgroup$ – erdal.karaca Jul 13 '18 at 17:28
1
$\begingroup$

Because the softmax layer ensures that the outputs can be interpreted as probabilities: it ensures that every output is between 0 and 1, and the outputs sum to 1. Without the softmax, you can't interpret them that way.

We want to interpret them as probabilities, both so that we can get confidence scores for the output of the classifier (the classifier doesn't just say "dog", it says 96% confidence it is a dog, 1% it is a bird, etc.). More importantly, the standard loss function, the cross-entropy loss, requires that the outputs be interpretable as probabilities. Empirically, using the cross-entropy loss (and a softmax layer) leads to good results for classification tasks -- better than other options.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.