1
$\begingroup$

In the context of feature relevance, I am trying to understand the meaning of the correlation method for feature selection. Can somebody please explain if the following results of the correlation coefficients arise, then should I take that feature? The rule is to select the features for which corrcoeff values are greater than 0.5. Please correct me if wrong. The way I am calculating is using Matlab's corrcoeff(target,feature) where target and feature are vectors

Case1: corrcoeff returns NaN values --

Nan Nan
Nan  1

Should the feature be selected since the value is greater than 0.5?

Case2: corrcoeff returns 0 values

0 0
0  1

In this case, I should reject the feature.

Case3:

-0.3 0
0    -0.3

Negatively correlated but absolute values less than 0.5, so reject the feature

Case4: What if there is no linear relationship at all in which case corrcoeff will not work. How do I know if there is no linear relationship and in that case how to do feature selection; is there any other function or technique?

$\endgroup$
  • $\begingroup$ Just some points, correlation matrix shouldn't be returned with NaN's, you might be doing wrong in some cases. In the third case, those -0.3 are the variance of first and second variable and 0 are the correlations, $\endgroup$ – Fatemeh Asgarinejad Jul 19 at 2:45
1
$\begingroup$

The idea is quite simple: If you find features that are correlated with your target variable but uncorrelated with each other, it will give you information with which you can predict your target variable.

The threshold of 0.5 is chosen by a person and should be changed depending on your data. For example, if you find 10 features that are uncorrelated with each other but all have a 0.3 absolute correlation coefficient with the S&P 500 than you found the stock market equivalent of the Holy Grail.

Also, as you already mentioned, the correlation coefficient only captures linear correlations. This could cause you to exclude strongly related features just because they have a more complex relationship to your target variable.

Therefore, I would strongly advise against this method. Instead, I recommend plotting the data in pairplots.

$\endgroup$
  • $\begingroup$ Thank you for your answer. I looked into the pairplots but it is not clear how to determine for linear correlation. Can you please explain or mention some tools to check for linearity? $\endgroup$ – Sm1 Jul 19 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.