3
$\begingroup$

Is it ok to use negatively correlated features in data modeling? Say I have features A and B that have a correlation coefficient of 0.2 and features C and D with -0.2 correlation coefficient, is it fine to use features C and D in the model, since they have a low negative correlation? Also, does this have different effects on a regression vs. classification problem?

$\endgroup$

2 Answers 2

3
$\begingroup$

Negative correlations are just as valid and useful as a positive correlation.
In your example, the 0.2 correlation and the -0.2 correlation have equal value in your model. A negative correlation just means that as one value goes up, the other goes down.
Also, the closer to 1 for a positive correlation and closer to -1 for a negative correlation, the more useful it will be for a modelling algorithm.

For most algorithms, the independent variables do not have to be un-correlated to be useful in a model. Most models will handle the cross-correlation between features, and in some cases, dropping one of them could actually be detrimental, possibly losing some information that would have been useful to the model.
Usually we drop features if we have too many of them, they are too sparse, or if our feature to row ratio is too high.
Both of these facts apply for Classification and Regression equally.

$\endgroup$
2
  • $\begingroup$ Closer the correlation coefficient is to 1 or -1 between the independent and dependent variable, the better right? But we want lower correlation between the independent variables. $\endgroup$
    – user100552
    Jul 14, 2020 at 14:05
  • 1
    $\begingroup$ Yes, your first statement is correct. To address your second statement: For most algorithms, the independent variables do not have to be un-correlated to be useful in a model. Most models will handle the cross-correlation between features, and in some cases, dropping one of them will actually be detrimental, possibly losing some information that would have been useful to the model. Both of these facts apply for Classification and Regression equally. Usually we drop features if we have too many of them, they are too sparse, or if our feature to row ratio is too high. $\endgroup$
    – Donald S
    Jul 14, 2020 at 14:21
3
$\begingroup$

There's no problem in having negative correlations, in fact, if you change C to -C the correlation will be positive and every machine learning model I know should be equivalent trained on a variable or the same one with the sign changed.

So no, in any supervised case, regression or classification, it doesn't matter if there are negative correlations among features, and it happens almost every time.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.