0
$\begingroup$

I have implemented the FP-growth algorithm and it works fine with this sample data:

r z h k p 
z y x w v u t s
s x o n r
x z y m t s q e
z
x z y r q t p

when I use

val fpgrowth = new FPGrowth().setItemsCol("items").setMinSupport(0.5).setMinConfidence(0.6)    

model.freqItemsets.show()

model.associationRules.show()

model.transform(dataset).show()

everything is displayed, but 1 or 2 items remain empty (model.transform(dataset).show() result:

+--------------------+----------+
|               items|prediction|
+--------------------+----------+
|     [r, z, h, k, p]| [t, y, x]|
|[z, y, x, w, v, u...|        []|
|     [s, x, o, n, r]| [t, y, z]|
|[x, z, y, m, t, s...|        []|
|                 [z]| [t, y, x]|
|[x, z, y, r, q, t...|       [s]|
+--------------------+----------+

Later I have tried a larger dataset with 88162 entries. However, I had to change the MinSupport to 0.01 in order to even get any results. The model.associationRules.show() diplayed something but the model.transform is empty for all items.

+--------------------+----------+
|               items|prediction|
+--------------------+----------+
|[0, 1, 2, 3, 4, 5...|        []|
|        [30, 31, 32]|        []|
|        [33, 34, 35]|        []|
|[36, 37, 38, 39, ...|        []|
etc.................

My questions is why is this happening with a large dataset and if I would like to recommend (something like: "this item is frequently bought together with") would I access the corresponding prediciton of that item ?

Thank you very much in advanced.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.