What is the difference between Linear Regression and Generalized Linear regression of degree 1? because linear regression uses ordinary least square method to find the best fit but GLM uses least squares approximation for finding the best fit I wonder whether the best fit we get from both is same for degree 1

  • $\begingroup$ Could you please explain what you mean be "degree 1"? Your use of this term makes me wonder if you're talking about GLMs at all and not comparing a simple linear regression ($y=a+bx$) to a multiple linear regression that happens to have only one predictor, which is exactly the same as simple linear regression. $\endgroup$ – Dave Feb 28 at 12:11

Linear regression, or Ordinary Least Squares (OLS) regression, is the foundation for generalized linear regression. I am not sure what you mean by "degree 1."

From Generalized Linear Models by John P. Hoffman: "...generalized linear models...[refer] to a class of regression models that allows us to generalized the linear regression approach to accomodate many types of dependent variables. We no longer have to assume that the outcome in our regression model is a continuous and normally distributed variable."

OLS is "generalized" to other applications via link functions. A link function is a transformation of the OLS equation so that the prediction is in an acceptable form.

Here is a list from wikipedia on some different link functions: https://en.wikipedia.org/wiki/Generalized_linear_model#Link_function

An example of this is logistic regression. The output in a logistic regression model is a value between 0 and 1, which the modeller can use to make a binary prediction. For example...

OLS gives you a single predicted value

OLS = β_1 X_1+β_2 X_2+⋯β_i X_i

The Logistic Function gives you the probability that Y = 1

Logistic Function = 1/(1+exp⁡[-1(OLS)])

The Logit Function gives you the log odds that Y = 1.

Logit Function = log_e(Logistic Function / (1-Logistic Function))

Within all of these equations is the original OLS equation. Each equation is a link function that modifies the OLS equation to give you the result in an output that is more applicable to your particular problem.

|improve this answer|||||
  • $\begingroup$ I have to voice some objection to your source saying that OLS assumes a Gaussian response variable. That is a typical assumption and is meaningful for parameter inference, but the Gauss-Markov theorem does not make this assumption, so the OLS solution $\hat{\beta}=(X^TX)^{-1}X^Ty$ gives the best linear unbiased estimator for a non-normal response. $\endgroup$ – Dave Feb 28 at 12:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.