I have a generated a set of imbalanced data: 70 samples of class 1 and 1000 samples of class 2. The target variable and all predictors are boolean.

I've made two predictors that should be relevant to the model:

  • "predictor1" is TRUE for for 60 out of the 70 class 1 samples and TRUE vor 300 out of the 1000 class 2 samples.
  • "predictor2" is TRUE for for 25 out of the 70 class 1 samples and TRUE vor 40 out of the 1000 class 2 samples.
  • "predictor2" is basically a subset of "predictor1" - All the TRUE values of "predictor2" coincide with TRUE values of "predictor1" (but that doesn't seem to matter).

All other predictors are randomly distributed.

My problem is: When I apply the Relief Algorithm implemented in the R CORElearn package "predictor2" is given the same importance as the random variables. The same is true for other weighted options of Relief in that package.

I'm confused... By inspection "predictor2" should also be relevant, for example for a decision tree construction. Calculation of the information gain confirms that. So why does Relief miss this relevant feature?

testdata.txt

testdata = read.csv("testdata.txt", header = TRUE, sep = ";",check.names=TRUE)

library(CORElearn)

#class_counts
colSums(subset(testdata,testdata$target==1))
colSums(subset(testdata,testdata$target==0))

# feature importance Relief
attrEval('target', testdata, estimator="Relief")

# Information gain for each feature
attrEval('target', testdata, estimator="InfGain")

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.