# Confusion regarding the Working mechanism of Activation function

For binary classification irrespective of the model used, the sigmoid function is a good choice for output layer because the actual output value ‘Y’ is either 0 or 1 so it makes sense for predicted output value to be a number between 0 and 1.

My confusion is that is there a binary step function in the output layer which squashes the values of the linear combination of weights and and inputs to 0 or 1? Does classification means always applying a thresholding function on top of a linear or non-linear function which is in the hidden layer?

Say the predicted output value is 0.75 and actual Y is 0. Then, how is 0.75 converted to 1? The loss function would calculate the error as actual - predicted = 0-0.75 = -0.75

Can somebody please explain the math or point out some links where the working steps are shown? Thank you.

I think following picture by Rubens Zimbres can express how this process will be done except loss function part:

Is there a binary step function in the output layer which squashes the values of the linear combination of weights and and inputs to 0 or 1?

Answer : Yes if you consider following example shows how Sigmoid squishes values to be between 0 and 1:

Does classification means always applying a thresholding function on top of a linear or non-linear function which is in the hidden layer?

Say the predicted output value is 0.75 and actual Y is 0. Then, how is 0.75 converted to 1? The loss function would calculate the error as actual - predicted = 0-0.75 = -0.75

Answer: Loss function and calculation of error for classification-based NN is not just by computation of $$Y_Actual - Y_prediction$$

but It is used normally by combination of Softmax activation function and cross-entropy as follow:

See the whole training process Multinomial Logistic Classification $$D(S(wx+b),L)$$

• Awesome! thank you for the illustration and point wise explanation.
– Sm1
Jun 18, 2019 at 0:54
• One last question -- why do we need sigmoid and not just a binary threshold function for the last output layer in classification to 0 and 1?
– Sm1
Jun 18, 2019 at 2:22