I have a generated a set of imbalanced data: 70 samples of class 1 and 1000 samples of class 2. The target variable and all predictors are boolean.
I've made two predictors that should be relevant to the model:
- "predictor1" is TRUE for for 60 out of the 70 class 1 samples and TRUE vor 300 out of the 1000 class 2 samples.
- "predictor2" is TRUE for for 25 out of the 70 class 1 samples and TRUE vor 40 out of the 1000 class 2 samples.
- "predictor2" is basically a subset of "predictor1" - All the TRUE values of "predictor2" coincide with TRUE values of "predictor1" (but that doesn't seem to matter).
All other predictors are randomly distributed.
My problem is: When I apply the Relief Algorithm implemented in the R CORElearn package "predictor2" is given the same importance as the random variables. The same is true for other weighted options of Relief in that package.
I'm confused... By inspection "predictor2" should also be relevant, for example for a decision tree construction. Calculation of the information gain confirms that. So why does Relief miss this relevant feature?
testdata = read.csv("testdata.txt", header = TRUE, sep = ";",check.names=TRUE) library(CORElearn) #class_counts colSums(subset(testdata,testdata$target==1)) colSums(subset(testdata,testdata$target==0)) # feature importance Relief attrEval('target', testdata, estimator="Relief") # Information gain for each feature attrEval('target', testdata, estimator="InfGain")