I'm self learning data science so bear with me as I try and make my question as clear as possible. Lets assume I have a dataset of a dependent variable y and some independent variables x1, x2 and x3 etc. Is the correlation between these variables simply a mathematical calculation? I mean if i ran a correlation test in R for example and the test came back with no correlation between the variables is that necessarily true? What if I knew or suspected that one of the variables is correlated to another how would I define that in my model or would relying on a correlation test be sufficient.


Yes, correlation is a mathematical concept, and it as well known as Pearson correlation. This is probably the one you are obtaining in your analysis.

However, there are other correlation analysis you can try in order to be totally sure about your result. The most famous ones out of Pearson are Kendall correlation and Spearman correlation.

In addition, sometimes relation between variables is less evident than a correlation analysis. For example, if the variables are time-related, one may have a lag over the other. For example, in finances it is known that the GDP is correlated with a lag of some months of the Employment Rate. It is part of a good Data Science knowledge to have a business knowledge, an intuition of why the variables may be related in some way and try to find how to prove that this non evident dependence exists.

Edit: I noticed you put R as a tag. You can study as well libraries polycor and ggm to learn about polychoric correlations and partial correlations, although the may not apply in your case.

  • 1
    $\begingroup$ It's worth remembering that correlation is only valid for continuous features. For pairs of categorical features, you should also look at Chi-squared / Cramer's V statistics. $\endgroup$
    – bradS
    Apr 6 '18 at 11:19

If you suspect that adding another variable does not improve the model, just leave it out, run the updated model again and check, for instance, the adjusted R-square value.

The anova() function takes two models and compares them more formally.

See this set of comprehensive answers to "What are modern, easily used alternatives to stepwise regression?" on the Cross-Validated site.

Good models give you insight into your problem (there may be several that do this).

(From R.Cotton (2011) Learning R book p 271)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.