10
$\begingroup$

In simple terms, what are the assumptions of Linear Regression?

I just want to know that when I can apply a linear regression model to our dataset.

Improve this question
$\endgroup$
1
  • $\begingroup$ you may mark the answer accepted if it is good enough for you. It has been here for sometime and you have not marked it yet. $\endgroup$
    – TwinPenguins
    Commented Dec 5, 2018 at 8:56

1 Answer 1

Reset to default
14
$\begingroup$

There are three major assumptions (statistically strictly speaking):

  1. There is a linear relationship between the dependent variables and the regressors (right figure below), meaning the model you are creating actually fits the data. enter image description here

  2. The errors or residuals of the data are normally distributed and independent from each other. enter image description here

  3. Homoscedasticity. This means the variance around the regression line is the same for all values of the predictor variable. enter image description here

Update 2:: Multicollinearity is not an assumption, but it is rather a sanity check especially if interpretability of the model is important (thanks Ricardo Cruz for the comment). Multicollinearity occurs when the independent variables are not independent from each other. Multicollinearity between explanatory variables which can leads to less stable parameter fits (thanks KT. for pointing this out)). There are tests like correlation matrix (Pearson's Bivariate Correlation), Variance Inflation Factor that can be used to verify this.

Improve this answer
$\endgroup$
7
  • 2
    $\begingroup$ Strictly speaking, 3 is not a direct assumption of the model. It can, however, be a nuisance as collinearity of the inputs leads to less stable parameter fits. $\endgroup$
    – KT.
    Commented Jun 1, 2018 at 10:54
  • $\begingroup$ Could not agree more! $\endgroup$
    – TwinPenguins
    Commented Jun 1, 2018 at 10:57
  • $\begingroup$ Then you could put 2. and 4. together into a single simple statement that "errors are independent of the input, i.i.d. normal random variables". This leaves with two assumptions, which correspond exactly to the probabilistic formula of the linear model. $\endgroup$
    – KT.
    Commented Jun 1, 2018 at 10:57
  • 1
    $\begingroup$ "which can leads to less stable parameter fits" - for the newbies, you guys should add that this is only a concern if you want to interpret the parameters. Just because the parameters aren't stable, it does not mean that the model itself is not stable, and that its predictions are inaccurate. Many people are only concerned about predictability, not interpretability, in which case multicolinearity is not a concern. $\endgroup$
    – Ricardo Cruz
    Commented Jun 2, 2018 at 21:24
  • 1
    $\begingroup$ Can you explain Homoscedasticity a little better with an example? It's not clear. You have marked one of my questions as a duplicate on which I was looking for a better view of it. Can you explain? $\endgroup$
    – Sai Kumar
    Commented Dec 19, 2018 at 8:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.