7
$\begingroup$

In simple terms, what are the assumptions of Linear Regression?

I just want to know that when I can apply a linear regression model to our dataset.

$\endgroup$
  • $\begingroup$ you may mark the answer accepted if it is good enough for you. It has been here for sometime and you have not marked it yet. $\endgroup$ – TwinPenguins Dec 5 '18 at 8:56
14
$\begingroup$

There are three major assumptions (statistically strictly speaking):

  1. There is a linear relationship between the dependent variables and the regressors (right figure below), meaning the model you are creating actually fits the data. enter image description here

  2. The errors or residuals of the data are normally distributed and independent from each other. enter image description here

  3. Homoscedasticity. This means the variance around the regression line is the same for all values of the predictor variable. enter image description here

Update 2:: Multicollinearity is not an assumption, but it is rather a sanity check especially if interpretability of the model is important (thanks Ricardo Cruz for the comment). Multicollinearity occurs when the independent variables are not independent from each other. Multicollinearity between explanatory variables which can leads to less stable parameter fits (thanks KT. for pointing this out)). There are tests like correlation matrix (Pearson's Bivariate Correlation), Variance Inflation Factor that can be used to verify this.

$\endgroup$
  • 1
    $\begingroup$ Strictly speaking, 3 is not a direct assumption of the model. It can, however, be a nuisance as collinearity of the inputs leads to less stable parameter fits. $\endgroup$ – KT. Jun 1 '18 at 10:54
  • $\begingroup$ Could not agree more! $\endgroup$ – TwinPenguins Jun 1 '18 at 10:57
  • $\begingroup$ Then you could put 2. and 4. together into a single simple statement that "errors are independent of the input, i.i.d. normal random variables". This leaves with two assumptions, which correspond exactly to the probabilistic formula of the linear model. $\endgroup$ – KT. Jun 1 '18 at 10:57
  • 1
    $\begingroup$ "which can leads to less stable parameter fits" - for the newbies, you guys should add that this is only a concern if you want to interpret the parameters. Just because the parameters aren't stable, it does not mean that the model itself is not stable, and that its predictions are inaccurate. Many people are only concerned about predictability, not interpretability, in which case multicolinearity is not a concern. $\endgroup$ – Ricardo Cruz Jun 2 '18 at 21:24
  • $\begingroup$ Can you explain Homoscedasticity a little better with an example? It's not clear. You have marked one of my questions as a duplicate on which I was looking for a better view of it. Can you explain? $\endgroup$ – Sai Kumar Dec 19 '18 at 8:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.