0
$\begingroup$

I'm working on a classification project. I have many rows, containing many binary attributes, some of which are often appearing together, exactly like what we can encounter in the Market Basket problem (in which you can, for example identify, that if you buy 'Milk' to a supermarket, you also have a more than random chance to buy 'Eggs').

My idea is then to take my target as an attribute, extract best Item-set containing my target (so having Target=1, exactly like my previous example if I wanted to make a model to predict whether the client will buy eggs or not).

Then, looking at the fit of each item-set, I'll have the info of which groups of products have best chances to make my target be 1.

Even though the method looks good, I have a problem. I'm wondering if there's a way to only extract item-sets with target on them.

All articles I found about making classification based on association rules explain that one of the big downside of the method is time taken, because it has to find all item-sets, and after removing all the ones that are not including target.

Is there a way, using item-set properties, to specify the algorithm (Apriori, FP-Growth, etc.) to only calculate Item-sets including Target, and not waste time calculating all item-sets?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

If I understand correctly you don't have your target and want to create it using associacion rules. Fp-growth algorithm already reduces itemset checks e.g. based on the fact that if a certain itemset doesn't match selected threshold any other superset of this itemset won't match it either. For this reason it is several times faster than apriori. If you want to create your target using associacion rules and then train a model to predict this target you need let the algorithm go over each example and let it discard each itemset not matching your threshold. What you can do to speed up the computation is:

  1. Use FP-Growth on a distributed system.
  2. Use even more restrictive condititons to discard itesets faster.
  3. Learn association rules only on a subset of items for example top 1000 bestsellers.
Improve this answer
$\endgroup$
2
  • $\begingroup$ In fact I have my target. For example I have products A, B, C, ..., X, and a binary target Y (for example, Y being "The Client is more than 30 yearsold"). What I want to do is take Y as a product (bought if valued one) and find itemsets including Y (for example, itemset B, G, H, Y exists. With this info, I know the group of products {B, G, H} are often linked to the fact my client is more than 30). Now, my issue is that FP-Growth will calculate all itemsets (including, for example {A, J, U}, which has no link with Y, so no interest on my problem). $\endgroup$
    – Adept
    Sep 8, 2020 at 13:23
  • $\begingroup$ In fact, I should use 2 variables Y_0 and Y_1, and find itemsets {FewProducts, Y_1} where {FewProducts, Y_0}'s fit a certain % times lower $\endgroup$
    – Adept
    Sep 8, 2020 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.