4
$\begingroup$

A website says Correlation refers to an increase/decrease in a dependent variable with an increase/decrease in an independent variable. Collinearity refers to two or more independent variables acting in concert to explain the variation in a dependent variable.Could someone clarify the terms ?

$\endgroup$
  • $\begingroup$ If you feel like your question has been answered please accept the respective answer so the question does not remain open (same goes for your other questions on this stack which are all open). $\endgroup$ – Sammy Jul 25 at 11:18
  • $\begingroup$ It concerns me that you’ve accepted an answer that gives the wrong definition of correlation. $X$ and $X^2$ are uncorrelated, despite the obvious relationship. (Assume $X_\sim N(0,1)$, since we can make them correlated for Bernoulli trials, for example, and perhaps for some other distributions.) $\endgroup$ – Dave Jul 26 at 15:15
0
$\begingroup$

Collinearity usually refers to any linear relationship or association between 2 or more features.

Correlation and correlated are more general, and can refer to any type of relationship between features and responses, including log, exponential and linear associations.

The word "correlation" is a noun. And its strength is measured by specific formula that depends upon the data-type and assumptions such as parametric or non-parametric.

The word "correlated" is adjective and indicates loose association between two variables i.e. it does not indicate causal relationship.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ In statistics, correlation specifically means a linear association. For example, points symmetrically arranged on a parabola have zero correlation. $\endgroup$ – Dave Jul 24 at 11:07
  • $\begingroup$ Two variables e.g. height and weight that are related (co-linear) and determine e.g. strength of an indivual(dependent variable). is the relation between two variables linear or co-linear. $\endgroup$ – Subhash C. Davar Jul 24 at 11:15
  • 1
    $\begingroup$ @Dave, I disagree that correlations are only linear in nature. For example, there are non-linear correlation metrics, such as Spearman's rank correlation coefficient, also Kendall's tau correlation coefficient. On a side note, it is usually better to transform correlated variables to have a linear relationship, as there are many more tools available for linear correlations. $\endgroup$ – Donald S Jul 24 at 11:41
  • $\begingroup$ “Correlation” without further specification is linear. $\endgroup$ – Dave Jul 24 at 11:44
  • $\begingroup$ @Subhash, your example of height and weight is 2 collinear variables that may have a linear relationship or dependence. Linear describes the relationship between the 2 variables, but the 2 variables are described as collinear. $\endgroup$ – Donald S Jul 24 at 11:48
1
$\begingroup$

Pearson correlation is the usual correlation when nothing further is specified and specifically refers to linear association.

$$\rho_{XY}=\dfrac{cov(X,Y)}{\sigma_X\sigma_Y}$$

In the world, people use “correlation” to mean any kind of association, but this is wrong from the standpoint of statistics. Arrange points symmetrically on a parabola and run them through that equation; you’ll get zero correlation, despite the obvious relationship.

There also is Spearman correlation, which does Pearson correlation on the ranks of the values.

If the points are $(0,1)$, $(2,4)$, $(3,3)$, the Spearman correlation is calculated by converting the $x$-values to their ranks and the $y$-values to their ranks: $(1,1)$, $(2,3)$, $(3,2)$. Then run the transformed points through the usual equation for (Pearson) correction.

To separate “correlation” and “correlated”, the former is a noun while the latter is an adjective. If there is “correlation” between two variables, they are “correlated”.

Collinear seems to come up in the context of regression and refers to predictor variables that are correlated. The related “multicollinear” means multiple regression predictors that have a linear relationship with another predictor, as if you could regress one predictor on some of the others and get decent accuracy. “Multicollinearity” seems to be the more common term to use when we talk about related predictors, as “collinear” variables strikes me as perfectly related with a correlation of $1$ (think of measurements in both meters in kilometers), while multicollinearity, to me, does not mean a perfect predictive ability unless “perfect” multicollinearity is specified.

“Collinear” and “multicollinear” are adjectives; “collinearity” and “multicollinearity” are the nouns.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Your example about the parabola is a serious misunderstanding of what the Pearson correlation coefficient measures... Of course you get a zero LINEAR correlation, even though there is a well defined non linear relationship $\endgroup$ – German C M Jul 26 at 14:56
  • $\begingroup$ That’s why we get zero Pearson correlation, despite the obvious relationship. $\endgroup$ – Dave Jul 26 at 15:09
  • $\begingroup$ Correlation misses the relationship. $\endgroup$ – Dave Jul 26 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.