What are some advanced or basic methods most used by data scientists/ML Engineers to detect collinearity (or) multicollinearity between features?


One way to measure multicollinearity is the Variance Inflation Factor (VIF), which assesses how much the variance of an estimated regression coefficient increases if your predictors are correlated.

If no factors are correlated, the VIFs will all be 1.

If the VIF is greater than 1, the predictors may be moderately correlated.

A VIF between 5 and 10 indicates high correlation that may be problematic.

And if the VIF goes above 10, you can assume that the regression coefficients are poorly estimated due to multicollinearity

If multicollinearity is a problem in your model the solution may be relatively simple. Try one of these:

Remove highly correlated predictors from the model. If you have two or more factors with a high VIF, remove one from the model. Because they supply redundant information, removing one of the correlated factors usually doesn't drastically reduce the R-squared. Consider using stepwise regression, best subsets regression, or specialized knowledge of the data set to remove these variables. Select the model that has the highest R-squared value.

Use Partial Least Squares Regression (PLS) or Principal Components Analysis, regression methods that cut the number of predictors to a smaller set of uncorrelated components.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.