1
$\begingroup$

I know that it is better to remove correlated features, but what if the feature correlates with the target label? So, it gives the most information for prediction? Do I need to remove features that correlate with the target label?

$\endgroup$
0
$\begingroup$

The idea behind removing features that correlate with each other is that, by increasing the independence of each feature in an observation a better model can be created. There is another thread that goes into further detail on the reasons why here.

However, the odd part of your question is that you are comparing the correlation between the dependent and independent variables. This type of comparison is usually done after a model is constructed in order to prevent to assess the predictive strength of the model. To compare the target label, the label you wish to predict, with the other variables before this is premature and will likely result in weakening your model.

When testing for correlated features, it is best that you first remove the target label, before check for the correlations and determining which to remove if any. Then if you want to remove features further you can perform principal component analysis on the data. From there you can run the data through say a logistic regression model and select the most important features from there, like mentioned here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.