After performing association rule mining, (using mlxtend library), I have gotten a pandas dataframe containing all the association rules. This contains too many rules, therefore we must reduce the number.
For an association rule $R: X \implies Y$
For any $Z \subset X$
If any rule $R': X - Z \implies Y$ exists, such that $$confidence(R) \pm \alpha = confidence(R')$$
Then remove $R$
I tried to implement that logic directly into python as such
def filter_useless(rules): shadow = rules def remove_one(tup): x , y, conf= tup['antecedents'] , tup['consequents'], tup['confidence'] for s in powerset(x): res = rules[ (rules['antecedents'] == s) \ & (rules['consequents'] == y) \ & (abs(rules['confidence'] - conf) <= 0.1)] if not res.empty: return False return True return shadow[shadow.apply(remove_one, axis=1)]
However, this didn't filter any rules. What am I doing wrong?