# Can you suggest a better to organise the data to generate frequent itemsets?

I have a data of a bag of words in a document. The data has 3 coulumns : document number, word number, count of the word in the number. I am supposed to generate frequent itemsets of particular size.

I thought I will make list of all words that appear in a document. Create a table of this list. Then generate frequent itemsets using Mlxtend or Orange . But it doesn’t seem to be efficient.

If the size is reasonable (i.e. not too many documents and not too many words in a document), you could try to build a map for each possible itemset, for instance like this:

// Assuming data is an array of size N containing all the documents
// clusters is a map associating each itemset with a set of documents
for i=0 to N-1
for j=i+1 to N-1
group = overlap(data[i], data[j])
add data[i] to the set clusters[group]
add data[j] to the set clusters[group]


An alternative version if the number of different values and size of the sets allow it and/or if it's possible to precompute the itemsets of interest:

for i=0 to N-1
for every subset S of data[i]
add data[i] to the set clusters[S]