4
$\begingroup$

Once I have been asked how would I calculate correlation between two time series. Since I am new to data science I answered: "I would just calculate the Pearson correlation coefficient". That wasn't a good answer since as demonstrated in the figure below, dependency between two variables may not be linear and the Pearson coefficient can be close to $0$ (parabola, circle). I have seen people on Kaggle always starting with correlation matrix and discarding data that are not correlated.

My question is: is Pearson coefficient always a good measure of correlation between variables and we should always rely on it? Can you give practical example when some variable in your problem was important, but Pearson indicated there is hardly any correlation?

Artificially generated data

Improve this question
$\endgroup$

3 Answers 3

Reset to default
3
$\begingroup$

As you show, Pearson coefficient is clearly not a good measure of how variables depend on each other. A better measure is distance correlation. A nice property of distance correlation is that distance correlation 0 implies independence. An easy example that can happen in real life is that a variable is the square of another variable, as you show in your first picture. In this case, the Pearson correlation will be 0, but the distance correlation will be around 0.5. I think it is a mistake for Kagglers to choose variables in terms of Pearson correlations.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for answer @David Masip! The example with quadratic dependency would be good if I have both arms of the parabola. If only one arms is visible (lets say I have positive values) then Pearson would be close to $1$. Distance correlation maybe a good way to decide if two variables are independent. $\endgroup$
    – WoofDoggy
    Commented May 30, 2018 at 11:14
  • $\begingroup$ I totally agree! $\endgroup$
    – David Masip
    Commented May 30, 2018 at 12:26
2
$\begingroup$

Can you give practical example when some variable in your problem was important, but Pearson indicated there is hardly any correlation?

Sure, when there is underlying nonlinear relation between two RVs. Pearson correlation investigates linear relations, therefore a low value does not exclude nonlinear relations.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Nice @pcko1. So should I perform some additional tests for nonlinear relations afterwards? $\endgroup$
    – WoofDoggy
    Commented May 30, 2018 at 11:15
  • $\begingroup$ Absolutely. I would personally check also the distance correlation, as suggested above :) $\endgroup$
    – pcko1
    Commented May 30, 2018 at 11:22
1
$\begingroup$

If you are considering time series only, you may have an the option to run a linear regression model, considering one variable as dependent.

If you can get a good R² and residual plot, with whatever transformation to make a linear model, then you may be able to assess there is a correlation between both.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Nonlinear relations are not exhibited by such analysis though. $\endgroup$
    – pcko1
    Commented May 30, 2018 at 9:30
  • $\begingroup$ What do you mean by exhibited ? $\endgroup$
    – Elliot
    Commented May 30, 2018 at 9:34
  • $\begingroup$ Demonstrated, revealed, etc. Having white residues in your correlation analysis does not mean that there aren't nonlinear relations between the variables. Therefore, you cannot imply that there is no dependency by this technique, as the OP asked. $\endgroup$
    – pcko1
    Commented May 30, 2018 at 9:37
  • $\begingroup$ True. But if you find white residues with a good R² for polynomial regression or regression with transformed data then it proves the relation between the two variables is well found with either your coefficients or transformation, and therefore exhibits it. $\endgroup$
    – Elliot
    Commented May 30, 2018 at 9:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.