How do we use a correlation score between two variables for analysing data?

I have a set of 20 features and need to predict 21st feature. Now is it necessary that correlation between any two features should be close to 1 ? If I have 2 features with corr score close to -1, then does this mean that they are contradicting and thereby decreasing the accuracy ?

So how do we use a correlation score in analysis ?


Correlation should be as less as possible between different features, because correlated features mean that those features are giving out same kind of information/trend for the predictor to learn. Thus only one of them is actually useful for prediction.

Keeping more number of uninformative features (correlated features) would result in degraded accuracy if your sample size is similar to you feature set size. Feature selection using Recursive Feature elimination or PCA etc. can help you reduce your feature set to optimal size.

We calculate correlation score in predictive analysis between features and Target variable. When using linear regression to model a data set, we first see if the plot between different features and target variable values follow an upward (+ve correlation) or downward trend (-ve correlation) and not scattered randomly. If such a relationship exists then regression modelling on the data would work well.

  • $\begingroup$ Thanks... you said that, first, pairwise correlation between target variable and a feature is examined... but what if target variable is categorical ( classification into one of the categories) ... in that case how to use correlation score $\endgroup$ – mach Oct 6 '15 at 13:43
  • $\begingroup$ If either of your feature or target is a categorical variable (also called nominal variable), then we don't find 'Correlation' between them, since correlation implies when one variable increases, an increase/decrease in the other is observed. Categorical variable doesn't possess numerical values, thus increase or decrease doesn't makes sense. We find association when either of your variable is categorical. $\endgroup$ – Santoshi M Oct 7 '15 at 6:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.