Skip to main content
Data Science

Return to Answer

added 4 characters in body
Source Link
noe
noe
  • 27.5k
  • 1
  • 49
  • 80

You should not remove features just because their correlation to the target is high. Such high correlation is a sign of potential target leakage. You should understand why their correlation is high.

If the highly-correlated feature is available at inference time and there is a causal relationship between it and the target, then the feature has high predictive power and you should keep it.

On the other hand, if the feature is not available at inference time or there is no causal relation at all, then this sounds like target leakage and you should remove it. This article has some nice examples of each case.

Another thing you may also pay attention tooto is correlation among features, which may be a bad thing depending on your model (i.e. linear model) and whether you want to interpret it (i.e. importance of each feature when predicting). This DS stackexchange question addresses such problem.

You should not remove features just because their correlation to the target is high. Such high correlation is a sign of potential target leakage. You should understand why their correlation is high.

If the highly-correlated feature is available at inference time and there is a causal relationship between it and the target, then the feature has high predictive power and you should keep it.

On the other hand, if the feature is not available at inference time or there is no causal relation at all, then this sounds like target leakage and you should remove it. This article has some nice examples of each case.

Another thing you may pay attention too is correlation among features, which may be a bad thing depending on your model (i.e. linear model) and whether you want to interpret it (i.e. importance of each feature when predicting). This DS stackexchange question addresses such problem.

You should not remove features just because their correlation to the target is high. Such high correlation is a sign of potential target leakage. You should understand why their correlation is high.

If the highly-correlated feature is available at inference time and there is a causal relationship between it and the target, then the feature has high predictive power and you should keep it.

On the other hand, if the feature is not available at inference time or there is no causal relation at all, then this sounds like target leakage and you should remove it. This article has some nice examples of each case.

Another thing you may also pay attention to is correlation among features, which may be a bad thing depending on your model (i.e. linear model) and whether you want to interpret it (i.e. importance of each feature when predicting). This DS stackexchange question addresses such problem.

Source Link
noe
noe
  • 27.5k
  • 1
  • 49
  • 80

You should not remove features just because their correlation to the target is high. Such high correlation is a sign of potential target leakage. You should understand why their correlation is high.

If the highly-correlated feature is available at inference time and there is a causal relationship between it and the target, then the feature has high predictive power and you should keep it.

On the other hand, if the feature is not available at inference time or there is no causal relation at all, then this sounds like target leakage and you should remove it. This article has some nice examples of each case.

Another thing you may pay attention too is correlation among features, which may be a bad thing depending on your model (i.e. linear model) and whether you want to interpret it (i.e. importance of each feature when predicting). This DS stackexchange question addresses such problem.