1
$\begingroup$

1- Would it always be beneficial to remove highly correlated features prior to training a model? If not, why not.

2- Would you perform One Hot encoding where applicable, prior to removing highly correlated features ? (taking into account a dummy variable may be highly correlated with another variable)

Thanks

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
2
$\begingroup$

(1) No. For example, if you have computer vision problems, then each pixel of the image is a feature. most neighboring pixels are highly correlated.

For example, see the following image from The HASYv2 dataset:

enter image description here

Improve this answer
$\endgroup$
2
  • $\begingroup$ Interesting. How about for classification/regression algorithms such as xgboost, random forest etc ? $\endgroup$
    – Gale
    Commented Mar 6, 2018 at 10:52
  • $\begingroup$ I haven't executed a comparative analysis of the performance of different classifiers with vs without the removal of highly correlated features. Only for CNNs on Computer Vision tasks I know for sure that you don't want to remove highly correlated features. $\endgroup$
    – Martin Thoma
    Commented Mar 6, 2018 at 11:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.