Relationship between log-odds and weighted sums in Logistic Regression

Question

I've read several articles/tutorials on Logistic Regression and I've come across this idea of log-odds being equal to the weighted sum of features.

i.e. if $p$ is the probability of a sample belonging to positive class(target variable: 1), $(1-p)$ is the probability of it belonging to the negative(target variable: 0). Then, for input features $x_1$, $x_2$, $x_3$ ... $x_n$ and weights $\theta_0, \theta_1, \theta_2... \theta_n$, they write

\begin{equation} log\bigg(\frac{p}{1-p}\bigg) = x_0 \theta_0 + x_1 \theta_1 + x_2 \theta_2 \dots x_n \theta_n \end{equation}

where $x_0$ is bias

Now I get what features are, I get what weighted sum of feature is, I know about the odds, I know log, what I cannot comprehend is, how are those two quantities equal? I've gone through plenty of online articles but they always say that these two are equal, but they don't explain how.

Answer 1

Ordinary regression, Logistic regression, Poisson regression are all examples of Generalized Linear Models (GLM).

A GLM can be broken down into 3 components:

• A random component: Target Variable $Y_i$ which is assumed to follow a distribution with $E(Y_i)$ and $Var(Y_i)$

• A systematic component: A linear combination of the features variables known as the linear predictor; $\gamma_i= \beta_{0} + \beta_1{x_1} \dots + \beta_n{x_n}$

• Link Component: The link function, $g(\cdot)$, connects the random component to the systematic component. If $E(Y_i) = u_{i}$, then the link function connects $\mu_{i}$ to $\gamma_{i}$ through some function $g(\cdot)$, and we then have that $g(\mu_i)= \beta_{0} + \beta_1{x_1} \dots + \beta_n{x_n}$. Moreover, $E(Y_i) = \mu_i = g^{-1}(\beta_{0} + \beta_1{x_1} \dots + \beta_n{x_n})$

The link function for linear regression is the identify link: $g(\mu_i) = \mu_i$

The link function for logistic regression is the logit link: $g(\mu) = log(\frac{\mu}{1-\mu})$. The logit link is used when $\mu$ takes values between 0 and 1 since the function maps to -infinity to infinity.

There are a lot details which I have skimmed over on purpose, GLM is a rich subject. I would recommend reading more about it, here are some references below:

References:

Categorical Data Analysis by Alan Agresti

An Introduction to Categorical Data Analysis

Hope this helps!