Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have a set of variables that I want to use for a regression or a classification problem. Having computed the correlation matrix of these variables, I discovered that some of them has an inter-variables Pearson correlation values as high as 1.
If the correlation between two features $x_1$ and $x_2$ is 1 that means that you can write $x_1 = c\cdot x_2 + a$. The only knew knowledge there is are those two constants, the individual values can be retrieved knowing this. I highly doubt there is anything a machine learning algorithm can learn from this and it is a fact that for some having this kind of correlation between features can hurt your performance quite a bit, so I would test it a bit but I would say it's very likely you can remove one of the two, and which one is not going to matter.
Yes and Yes.
Variance Inflation Factor is a common method for addressing your concerns.
https://en.m.wikipedia.org/wiki/Variance_inflation_factor
Correlation based feature selection is another approach.
https://en.wikipedia.org/wiki/Feature_selection#Correlation_feature_selection
