Why we prefer VIF if we can find multicollinearity from correlation matrix as well? What is the exact logic behind it.
Thanks for the help.
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
The correlation matrix is not a reliable measurement for multicollinearity because it only considers the pairwise effects. Unfortunately, multicollinearity is defined as:
Phenomenon in which two or more predictor variables in a multiple regression model are highly correlated,
Do you see the point? You'll need to consider the correlation with all other variables in your data set, not just 1-to-1 pairwise comparison.
VIF addresses the issue.
If you have 10000 rows and 100 columns which have all 100 1s and the rest 0s. Every row has a 1 and the rest 0, a classic one hot encoded matrix. The correlations between two random columns is -0.01010101, which means the correlation matrix has a diagonal of 1s and the rest is -0.01010101. However this matrix is perfectly multicolinear.