What is the difference between Linear Regression and Generalized Linear regression of degree 1? because linear regression uses ordinary least square method to find the best fit but GLM uses least squares approximation for finding the best fit I wonder whether the best fit we get from both is same for degree 1


Linear regression, or Ordinary Least Squares (OLS) regression, is the foundation for generalized linear regression. I am not sure what you mean by "degree 1."

From Generalized Linear Models by John P. Hoffman: "...generalized linear models...[refer] to a class of regression models that allows us to generalized the linear regression approach to accomodate many types of dependent variables. We no longer have to assume that the outcome in our regression model is a continuous and normally distributed variable."

OLS is "generalized" to other applications via link functions. A link function is a transformation of the OLS equation so that the prediction is in an acceptable form.

Here is a list from wikipedia on some different link functions: https://en.wikipedia.org/wiki/Generalized_linear_model#Link_function

An example of this is logistic regression. The output in a logistic regression model is a value between 0 and 1, which the modeller can use to make a binary prediction. For example...

OLS gives you a single predicted value

OLS = β_1 X_1+β_2 X_2+⋯β_i X_i

The Logistic Function gives you the probability that Y = 1

Logistic Function = 1/(1+exp⁡[-1(OLS)])

The Logit Function gives you the log odds that Y = 1.

Logit Function = log_e(Logistic Function / (1-Logistic Function))

Within all of these equations is the original OLS equation. Each equation is a link function that modifies the OLS equation to give you the result in an output that is more applicable to your particular problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.