I have a large amount of categorical and dummy variables (36) and I would like to remove a number of them based on their multicollinearity (or just collinearity). Instead of using Chi Square tests over and over again, are there any functions that can check for (multi)collinearity in my variables and return variables with multicollinearity (or collinearity)?
1
-
$\begingroup$ Did you try car::vif? Or usdm::vif? The VIF is the way to do it. $\endgroup$– Steven SlezakApr 16, 2018 at 4:59
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
In my work I usually use Normalized Mutual Information (NMI) to get an understanding of how "correlated" two categorical variables are.
Normalized Mutual Information is an information-theoretic measure that tells you how much information is shared by two variables. If NMI is close to 1, the two variables are very "correlated", while if NMI is close to 0 the two variables are "uncorrelated".
I wrote this function that computes the NMI between the first two variables in a data.table. It is quite fast thanks to data.table. Feel free to use it!
compute_mutualinfo <- function(df){
require(data.table)
require(entropy)
var1 = names(df)[1]
var2 = names(df)[2]
tmp_tab <- df[,.N,by=c(var1,var2)]
names(tmp_tab)[1:2] <- c('V1','V2')
cross_tab <- tapply(tmp_tab$N,list(addNA(as.factor(tmp_tab$V2),ifany = T), addNA(as.factor(tmp_tab$V1),ifany = T)), sum)
cross_tab[is.na(cross_tab)] <- 0
mi.plugin(cross_tab)/sqrt(entropy(df[,.N,by=var1,with=T][,N])*entropy(df[,.N,by=var2,with=T][,N]))
}
answered Apr 16, 2018 at 9:14
$\begingroup$
$\endgroup$
A variance inflation factor(VIF) detects multicollinearity in regression analysis. I learned that a VIF above 10 indicates multicollinearity and should be treated carefully.
Like Stephen mentioned the vif function is a part of the car library
