1- Would it always be beneficial to remove highly correlated features prior to training a model? If not, why not.

2- Would you perform One Hot encoding where applicable, prior to removing highly correlated features ? (taking into account a dummy variable may be highly correlated with another variable)



(1) No. For example, if you have computer vision problems, then each pixel of the image is a feature. most neighboring pixels are highly correlated.

For example, see the following image from The HASYv2 dataset:

enter image description here

  • $\begingroup$ Interesting. How about for classification/regression algorithms such as xgboost, random forest etc ? $\endgroup$ – Gale Mar 6 '18 at 10:52
  • $\begingroup$ I haven't executed a comparative analysis of the performance of different classifiers with vs without the removal of highly correlated features. Only for CNNs on Computer Vision tasks I know for sure that you don't want to remove highly correlated features. $\endgroup$ – Martin Thoma Mar 6 '18 at 11:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.