I have a DataFrame in python by using pandas which has 3 columns and 80.000.000 rows.

The Columns are: {event_id,device_id,category}.[ here is the first 5 rows of my df]

each device has many events and each event can have more than one category.

I want to run Apriori algorithm to find out which categories seem together.

My idea is to create a list of lists[[]]: to save the categories which are in the same event for each device. like: [('a'),('a','b')('d'),('s','a','b')] then giving the list of lists as transactions to the algorithm. I need help to create the list of lists.

If you have better idea please tell me because I am new in Python and this was the only way I found out.


There's your transactions LOL.

If you aren't limited to Python 2.7, I'd suggest Orange3-Associate which contains a frequent_itemsets() function based on FP-growth algorithm, which is orders of magnitude faster than Apriori.

  • $\begingroup$ Your code does not consider the events. For example, device_id=4 has 3 different events.I wand this 3 different events as 3 different transactions. I should use python, I can not move to Orange. $\endgroup$ – Ayn Mar 14 '17 at 22:45
  • $\begingroup$ In that case, df.groupby(['device_id', 'event_id'])... will group by unique device-event combinations. And Orange3-Associate is a Python package. But if it's for homework, you may need to code it yourself. $\endgroup$ – K3---rnc Mar 15 '17 at 2:05
  • $\begingroup$ Thank you for your help. can you tell me how can I give this list of lists to Orange? $\endgroup$ – Ayn Mar 15 '17 at 11:18
  • $\begingroup$ Please RTFM. $\endgroup$ – K3---rnc Mar 15 '17 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.