# Sigmoid function

How did we arrive at the sigmoid function for calculating probabilities?

Why not use some other function that " squashes " the values to lie between [0, 1]. Maybe even just normalise the values so they all add up to one.

• The derivative of this function is nice, making it invaluable for gradient descent. – Emre Mar 30 '17 at 4:42
• stats.stackexchange.com/questions/162988/… – SmallChess Mar 30 '17 at 5:09
• This question starts with a false premise. Many people use other linking functions. Logit is the most popular, but probit is very popular as well. – Ricardo Cruz Mar 31 '17 at 9:28

1. The most obvious idea is to let $p(x)$ be a linear function of $x$. Every increment of a component of $x$ would add or subtract so much to the probability. The conceptual problem here is that $p$ must be between $0$ and $1$, and linear functions are unbounded. Moreover, in many situations we empirically see “diminishing returns” — changing $p$ by the same amount requires a bigger change in $x$ when $p$ is already large (or small) than when $p$ is close to $1/2$. Linear models can’t do this.
2. The next most obvious idea is to let $\log p(x)$ be a linear function of $x$, so that changing an input variable multiplies the probability by a fixed amount. The problem is that logarithms are unbounded in only one direction, and linear functions are not.
3. Finally, the easiest modification of $\log p$ which has an unbounded range is the logistic (or logit) transformation, $\log (p(1−p))$ . We can make this a linear function of $x$ without fear of nonsensical results. (Of course the results could still happen to be wrong, but they’re not guaranteed to be wrong.)