1
$\begingroup$

I have a DataFrame in python by using pandas which has 3 columns and 80.000.000 rows.

The Columns are: {event_id,device_id,category}.[ here is the first 5 rows of my df]

each device has many events and each event can have more than one category.

I want to run Apriori algorithm to find out which categories seem together.

My idea is to create a list of lists[[]]: to save the categories which are in the same event for each device. like: [('a'),('a','b')('d'),('s','a','b')] then giving the list of lists as transactions to the algorithm. I need help to create the list of lists.

If you have better idea please tell me because I am new in Python and this was the only way I found out.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$ 
df.groupby('device_id')['category'].apply(list).tolist()

There's your transactions LOL.

If you aren't limited to Python 2.7, I'd suggest Orange3-Associate which contains a frequent_itemsets() function based on FP-growth algorithm, which is orders of magnitude faster than Apriori.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Your code does not consider the events. For example, device_id=4 has 3 different events.I wand this 3 different events as 3 different transactions. I should use python, I can not move to Orange. $\endgroup$
    – Ayn
    Mar 14, 2017 at 22:45
  • $\begingroup$ In that case, df.groupby(['device_id', 'event_id'])... will group by unique device-event combinations. And Orange3-Associate is a Python package. But if it's for homework, you may need to code it yourself. $\endgroup$
    – K3---rnc
    Mar 15, 2017 at 2:05
  • $\begingroup$ Thank you for your help. can you tell me how can I give this list of lists to Orange? $\endgroup$
    – Ayn
    Mar 15, 2017 at 11:18
  • $\begingroup$ Please RTFM. $\endgroup$
    – K3---rnc
    Mar 15, 2017 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.