2
$\begingroup$

i've been tying to wrap my head around logistic regression, the logit transform, and the sigmoid function.

logit transform: enter image description here

from what i understand, in practice all we want to do is maximize the following log likelihood (to get the sigmoid parameters that best fit the data): log likelihood enter image description here which involves some form of gradient descent to find the weights that maximize the above expression.

what i'm a bit hung up on is i don't see the logit transform anywhere here, ie, it looks like it isn't even used in the modeling process.

so is the logit transform actually ever computed in logistic regression? or is it just used to set up the problem, as a rationale for maximizing the log likelihood (rather than computing linear least squares)?

its a bit similar to the question below, i understand that the logit is the inverse of the sigmoid, and in that sense it is "used" in the modeling process, but i don't see the log odds anywhere in the modeling process. thanks,

What is the purpose of Logit function? At what stage of model building process this logit function is used?

$\endgroup$
2
$\begingroup$

Basically the Logit function is a series of transformations from ordinary regression. Transformations from ordinary regression are often called "link functions" It goes:

Ordinary Least Squares (OLS) -> Logistic Function -> Logit Function

You probably already know these equations but I am going to write them out anyway:


OLS gives you a single predicted value

OLS = β_1 X_1+β_2 X_2+⋯β_i X_i


The Logistic Function gives you the probability that Y = 1

Logistic Function = 1/(1+exp⁡[-1(OLS)])


The Logit Function gives you the log odds that Y = 1.

Logit Function = log_e(Logistic Function / (1-Logistic Function))


You are asking whether or not the logit function is ever actually used and the answer is yes, but you probably don't see it because a model summary will display the coefficients (betas) but not the equation that they're in. Your preferred language will certainly use these equations when estimating your coefficients, but there is no reason to show them to the user because the user is the one who specified the link function in the first place.

glm(y ~ x1 + x2, data = my.df, family = binomial(link = "logit"))
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.