In the context of multi-regression, I am wondering if there is a way to decompose $$VIF_i = 1/(1-R_i^2)$$ where $R_i^2$ is the r squared obtained from the regression of dependent variable = i and independent variables are all other factors.

I want to decompose $VIF_i$ or $R_i^2$ into individual factors to see how much each individual factor contributes to the $VIF_i$ or $R_i^2$

Someone recommended using the square of partial correlation coefficient and that value is linearly related to $R_i^2$. My undestanding is that partial correlation coefficient measures the correlation between two variables, holding the other variables constant. Is this a viable option?

  • $\begingroup$ Decomposing the R-squared (e.g. by looking at squared semi-partial correlations) and gain the information you are interested in is difficult. Imagine e.g. a situation with three highly correlated covariables $z_1$, $z_2$, $z_3$. How would you want to decompose the R-squared corresponding to the VIF of $z_1$ in contributions of $z_2$ and $z_3$? $\endgroup$
    – Michael M
    Aug 29 '18 at 16:27
  • $\begingroup$ @MichaelM well.. I do not know how to decompose the R-squared corresponding to the VIF of z1 in contributions of z2 and z3 and that's why I posted the question to get an answer. $\endgroup$
    – Jun Jang
    Aug 29 '18 at 16:47
  • $\begingroup$ datascience.stackexchange.com/a/64415/71442 $\endgroup$
    – Peter
    Jan 10 at 13:52

I suggest calculating R-squared and VIF for each permutation of variable combinations.

Also consider the fact that variable interactions may influence your R-squared and VIF values. Depending on what programming language you are using there are many ways to build interactions into your regression model, and you can iteratively test for effects from there.

  • $\begingroup$ From my understanding, Cook's Distance is used to identify an influential observation in regression. I am not trying to identify influential points. I want to identify influential "variables" (aka x's) $\endgroup$
    – Jun Jang
    Sep 4 '18 at 12:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.