# How to interpret correlation matrix?

I have built a correlation matrix to check multicollinearity in a regression model. But how to interpret this?

1. Can we say that there is a certain correlation value from which the independent variables related to this correlation should be removed from the model? Is this the value over 0.5?
2. Are there any better solutions than removing the conflicting variables (when correlation value is too big)? I tried normalization but it didn't help.
• You can try to apply PCA or TruncatedSVD to get/transform and select only the best and meaningfull features. Or maybee apply RFC and look for feature_importances_ attribute to help you decide which features have to be kept.
– Malo
Commented Jul 7, 2021 at 22:19
• Despite how much sense it makes at first, dropping variables has its problems.
– Dave
Commented Mar 8, 2022 at 10:52

Yes; selecting based on the correlation coefficient, which I'll call $$r$$, is a valid option. It doesn't necessarily have to be $$|r|>0.5$$, but keep in mind that the lower you go, the more likely you are to lose valuable information. You may also decide that you wish to eliminate a certain number of features, $$k$$, and choose these based on the $$k$$-highest correlation coefficients.
If the reason why you want to eliminate variables is because you're worried about redundancy between features harming your predictivity, I would consider eliminating pH and stopping there, since it correlates with so many other variables. If you simply don't want to deal with too many variables, perhaps start eliminating the ones that correlate with pH (but not pH). I would prioritize elimination based on what makes sense in the real world, especially if you do not have a lot of data samples (meaning some of those high $$|r|$$ could be influenced by small sample size). E.g., I'm guessing you weren't surprised by pH being correlated with fixed acidity, since pH is an indication of how acidic or basic a compound is.