What if one of the predictor variables is highly correlated with the target variable (say 0.9), what should we do? Should we drop it or keep it to build the prediction model(classification or regression)?
2
-
3$\begingroup$ Not really sure to understand the question, but why should we drop a variable that is useful to predict the answer ? it seems fairly smart to keep it so the model can use it to predict the correct answer. May you explain why you would drop it ? $\endgroup$– UbikuityCommented May 25, 2021 at 14:42
-
4$\begingroup$ Generally you want features that correlate highly with the target variable. However for prediction you need to be careful that: 1) the feature will truly be available at prediction time (i.e. there is no leakage), and 2) that the relationship is reasonably generalizable (i.e. not relying on quirks of the training data that will not generalize to the deployed model). $\endgroup$– GeoMatt22Commented May 25, 2021 at 14:45
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
You should keep it, the higher the correlation with the target variable - the better the feature. BUT - you should also make sure this correlation is "real", i.e. not due to data leakage.
(the answer was written using @GeoMatt22 and @Ubikuity comments.)
-
$\begingroup$ website defines "Correlation is a measure of the degree of linear association among a pair of variables." Correlation is "real", i.e. not due to data leakage. These are not correct interpretations. What is the formula you assume for computing correlation. $\endgroup$ Commented Jun 16, 2021 at 15:04
-
$\begingroup$ There is a term _" linear correlation" in statistics. Correlation is linear or "non-linear". $\endgroup$ Commented Jun 16, 2021 at 15:16