0
$\begingroup$

For a binary DNN, the output is $y_0 + y_1 = 1$ since they are the probability distribution, hence the sum must equate to 1. However, I've been told that $y_1$ is sufficient to represent the output of the DNN where:

$y_1 = \frac{e^{h_1}}{{e^{h_0} + e^{h_1}}}$

Hence, $h = h_1 - h_0$, why is it that $y$ which is the final output can now be represented as $y = \sigma(h)$, where $\sigma$ is the sigmoid function. Could anyone please explain why? How did it derive to the point where $y = \sigma(h)$?

$\endgroup$

1 Answer 1

0
$\begingroup$

In a binary classification problem you have only 2 classes, let's call them the negative and the positive class.
You only need to ouptut 1 number which corresponds to the probability of your input point to belong to the positive class.
The sigmoid activation function is good for that because it maps any input value to the range ]0,1[ which is what we want for a probability (it is not a real issue that 0 and 1 are excluded).

Since you have only one output number, it makes no sense to use a softmax activation.
Softmax activation is used in the multiclass problem where you must predict 1 of N classes where N is greater or equal than 3 and in this case the number of outputs is N (1 probability by class).
The softmax function makes that all your outputs sum to 1 and it amplifies the gap between high and low probabilities.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.