Log-odds has a linear relationship with the independent variables, which is why log-odds equals a linear equation.

What about log of probability? How is it related to the independent variables? Is there a way to check the relationship?


1 Answer 1


In logistic regression, it isn’t the case that the log-odds are linearly related to the features. We posit that such a relationship exists and then find the coefficients giving the best fit. That assumed linear relationship between the log-odds and the features might be an awful assumption, and that is why models like neural networks can be useful.

If you want to propose a binomial model with $\log(\mathbb E[Y\vert X])=X\beta$, feel free to do so. In fact, R has no trouble fitting such a model.

N <- 100
x <- runif(N, 1, 2)
y <- rbinom(N, 1, 0.5) 
L <- glm(y ~ x, family = binomial(link = log))
  • $\begingroup$ Why do we have to make an assumption about a linear relationship between the odds of success & the independent variables? Why can't we just fit a straight line/curve through the data points as per possibility & infer coefficients from it? And why don't we take log of probability, make a similar assumption on linearity & fit a line through the data? Do we reject probability for prediction & choose odds because we need a likelihood model for parameter estimation? $\endgroup$
    – Apoorva
    Commented Apr 22, 2022 at 3:30
  • $\begingroup$ @Apoorva 1) You might be interested in linear probability models. Nothing forces you to use the logistic link function. $\endgroup$
    – Dave
    Commented Apr 27, 2022 at 12:11
  • $\begingroup$ I think this isn't getting at why log-odds is the right choice, and why a log transformation of anything is necessary. It stems from what you think the distribution of the target it is - the prediction is binomial (0/1) with some probability p. Plot this data - every y value is 0 or 1. Fitting a line doesn't make sense. You will predict < 0 or > 1 in many cases. It turns out that regressing log odds is the choice consistent with Binomial(p) distribution of the target. $\endgroup$
    – Sean Owen
    Commented May 27, 2022 at 12:19
  • 1
    $\begingroup$ @SeanOwen Or maybe a probit link fits better. Maybe a Cauchit fits better. Maybe a linear probability model with an identity link function fits better. The logistic link isn’t a law. $\endgroup$
    – Dave
    Commented May 27, 2022 at 12:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.