1
$\begingroup$

What is the purpose of the logistic sigmoid function as it is used in logistic regression? Why does it need to be part of the hypothesis function h(x) ?

As I understand it, the logistic sigmoid function gives the probability that a certain input vector x is contained within a class C1 for a label y. In the binary-class case, it seems that if h(x) >= 0,5, we say that x belongs to one class, otherwise it belongs to the other. In the logistic regression model, our hypothesis function h(x) is of the form g(p^T * x), where p is the parameter vector (p^T is the transpose) and g is the sigmoid function.

Since the y-intercept of the logistic sigmoid is 0.5, saying that h(x) >= 0.5 is the same as saying p^T * x >= 0.

What I'm getting at is why do we need the logistic sigmoid function at all to define some threshold for separating the classes? Why not just let the hypothesis function be of the form h(x) = p^T * x, and claim that y = 1 if p^T * x >= 0? Why complicate things unnecessarily with the logistic sigmoid?

$\endgroup$
  • 1
    $\begingroup$ We do exactly what you are talking about with discriminative models. See SVMs for example. Logistic regression is a probabilistic model. $\endgroup$ – mrmcgreg Jun 8 '18 at 20:51
2
$\begingroup$

If you go and define the hypothesis function as h(x) = p^T * x, and claim that y = 1 if p^T * x >= 0, you are complicating the things even more.

This is due to the fact that during training time you will have to calculate the derivative of the function h(x). Now your function can lead to exploding or vanishing gradients depending on your input and initialization.

Logistic Sigmoid solves this problem and it also has a nice gradient g(x)(1 - g(x)). So rather that complicating things, sigmoid actually simplifies the calculation.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.