Log-odds has a linear relationship with the independent variables, which is why log-odds equals a linear equation.
What about log of probability? How is it related to the independent variables? Is there a way to check the relationship?
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
In logistic regression, it isn’t the case that the log-odds are linearly related to the features. We posit that such a relationship exists and then find the coefficients giving the best fit. That assumed linear relationship between the log-odds and the features might be an awful assumption, and that is why models like neural networks can be useful.
If you want to propose a binomial model with $\log(\mathbb E[Y\vert X])=X\beta$, feel free to do so. In fact, R has no trouble fitting such a model.
set.seed(2022)
N <- 100
x <- runif(N, 1, 2)
y <- rbinom(N, 1, 0.5)
L <- glm(y ~ x, family = binomial(link = log))
summary(L)
-
$\begingroup$ Why do we have to make an assumption about a linear relationship between the odds of success & the independent variables? Why can't we just fit a straight line/curve through the data points as per possibility & infer coefficients from it? And why don't we take log of probability, make a similar assumption on linearity & fit a line through the data? Do we reject probability for prediction & choose odds because we need a likelihood model for parameter estimation? $\endgroup$– ApoorvaCommented Apr 22, 2022 at 3:30
-
$\begingroup$ @Apoorva 1) You might be interested in linear probability models. Nothing forces you to use the logistic link function. $\endgroup$– DaveCommented Apr 27, 2022 at 12:11
-
$\begingroup$ I think this isn't getting at why log-odds is the right choice, and why a log transformation of anything is necessary. It stems from what you think the distribution of the target it is - the prediction is binomial (0/1) with some probability p. Plot this data - every y value is 0 or 1. Fitting a line doesn't make sense. You will predict < 0 or > 1 in many cases. It turns out that regressing log odds is the choice consistent with Binomial(p) distribution of the target. $\endgroup$ Commented May 27, 2022 at 12:19
-
1$\begingroup$ @SeanOwen Or maybe a probit link fits better. Maybe a Cauchit fits better. Maybe a linear probability model with an identity link function fits better. The logistic link isn’t a law. $\endgroup$– DaveCommented May 27, 2022 at 12:23