1
$\begingroup$

I have a set of independent variables and I am calculating the correlation matrix between them using the Pearson Correlation Coefficient in Python. A part of the matrix looks like this:

Dataset Correlation Matirx

From this matrix, suppose I want to find out the strongly correlated components between the variable NoOfDoors and the rest(Symboling...Compression Ratio). The process I have adopted is that I have taken the mean of that column(which is calculated as 0.039604) and based on that, I have only considered those values greater than 0.039604.

Based on that, the following variables have been selected as strongly correlated:

(Make, Aspiration, Wheel Base, Length, Width, Height, Curb Weight, Engine Type, Bore, Compression Ratio)

I want to ask, is this selection correct? If yes then is there an efficient way to do this? And if no, what is the correct way? Since I am new to this field, a well explained article would be appreciated. Thanks!

$\endgroup$
  • $\begingroup$ I would suggest to calculate the mean of the absolute value of that column, because a "very negative" value also means strong correlation, just in the opposite direction. You can also consider rank the entire correlation matrix by absolute value. $\endgroup$ – user12075 Jan 31 '19 at 7:08
2
$\begingroup$

There is no universal definition of "strongly correlated" and thus there is no "correct" answer to your question. One approach would be to perform a significance test to determine whether there is strong evidence that the correlations are nonzero. The mathematical details of the test are here, and the relevant Scipy API documentation is here.

However, "probably nonzero" may not be what you're looking for when you say "strongly correlated." If that's the case, you might want to consider why you're interested in finding the strongly correlated variables in the first place. Are you interested in identifying the strongly correlated features for a predictive problem? If so, you'll likely be better off skipping this exercise and starting to experiment directly with learning algorithms. Are you interested in determining what predictor variables cause changes in a certain response variable? In this case, you'll probably want to look into some kind of statistical regression model (probably starting with linear regression) and be very careful about all of the assumptions involved when drawing causal conclusions.

| improve this answer | |
$\endgroup$
1
$\begingroup$

Great answer by @StubbornTurtle.

To give a few more details see this answer. This talks about suppressor, mediators, confounders, and moderators. Suppressors are weakly correlated with a predictor but the suppressor's inclusion in the model makes the model stronger since it helps predict the target where another predictor is weak or as I was taught explain the variance.

All data sets do not have suppressors. However, by only including "highly" correlated, you may lose these variables and make your model weaker.

Second, we often want some predictors that are uncorrelated with each other if the predictors make the model "better", where better is defined by the problem the model is trying to solve. Uncorrelated predictors may be latching onto different signals and noises in the model. If the predictors are all highly correlated, they may be all explaining the same signal and containing the same noise.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.