0
$\begingroup$

I'm looking at datasets where the the attributes and the target class have a logical relationsships. All attributes and the target class are binary.

Here's an example: Neither feature#1 nor feature#2 have a significant correlation with the target class. But the conjunction feature#1 AND feature#2 is correlated with the target class.

Are there any Feature Selection Algorithms able to cope with situations like that? I'm thinking that frequent itemsets could be useful. It would be tremendously helpful, if anyone could point me to a related paper.

Cheers!

Improve this question
$\endgroup$
6
  • $\begingroup$ As features can be extracted from some models, there are many choices. $\endgroup$
    – Icyblade
    Feb 13, 2017 at 15:03
  • $\begingroup$ Sure, but there has to be a ranking of feature importance of some kind for that. For polynomial models those are usually the coefficient weights. For a decision tree one could use the information gain for each feature. But what can I do, when the features show a correlation to the target class only in conjunction with each other but not individually? That case is rather hard to represent with polynomial models oder DTs... $\endgroup$
    – AutoMiner
    Feb 13, 2017 at 16:04
  • $\begingroup$ I don't have the answer but any reliable way to do this will get computationally expensive pretty fast without some loose heuristics $\endgroup$
    – Jan van der Vegt
    Feb 13, 2017 at 16:55
  • $\begingroup$ I don't see why L_1 regularization shouldn't pick this up. $\endgroup$
    – Emre
    Feb 13, 2017 at 17:09
  • $\begingroup$ @Emre Because conjunctions require non-continous functions in a model. $\endgroup$
    – AutoMiner
    Feb 14, 2017 at 15:22

1 Answer 1

Reset to default
1
$\begingroup$

Emergent pattern mining could be useful for you. In classification, emergent patterns are subsets of features which differentiate one class from the others. Look at Example 1.1 in the paper. Two feature subests, denoted $X$ and $Y$, are EPs because $X$ has large support in one class and small support in the other, and vice versa for $Y$. The nifty thing about EP mining is that it considers subsets of features instead of individual features.

Mining patterns in this way could address your problem of feature1 and feature2 having no correlation with the target class alone, but the combination feature1 AND feature2 having high correlation with the target class.

Unfortunately, I don't see any open-source software implementations on the web. However, there are a number of papers out there which implement some sort of EP mining.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.