I'm doing my master thesis on Big Data Analytics. I'm trying to develop a algorithm to identify associations between some products in a supermakert. Imagine that I've this dataset:

Purchase_ID Product_ID  Purchase_Value
    1       2           4.5
    1       3           1.2 
    2       3           1.4 
    2       1           3.5
    2       2           7.3
    3       2           0.5
    3       3           1.0

What I want to conclude is that: "Every people that by Product_ID 2 also buy 3"

Anyone knows If exists any code algorithm available to use in Spark Mllib? I already search on internet but I didn't found anything...

Anyone can help me?

Many thanks!

  • $\begingroup$ I'm having troubles understanding the purchase_value column as it is inconsistent considering the product_id. Also if we forget a bit about big data and spark, your problem as you define it can be associated to any of the following : recommendation system, similarity measures, frequent pattern matching. Thus making your question quite broad to answer. Would you care reviewing your question for the matter so we can try to help more efficiently ? $\endgroup$ – eliasah Aug 26 '16 at 15:41
  • $\begingroup$ hi eliasah, many thanks for your help :) The purchase value just shows the value of the product in that specific purchase it can be manipulate by quantity or promotions. What I'm trying to get is a algorithm that can anlyze the list of items by each purchase and can extract the products that are purchased together. $\endgroup$ – João_testeSW Aug 26 '16 at 15:47

This sounds like a classic frequent pattern mining problem, where you're trying to find sets of items that are frequently found together within users' sets of purchases.

Start here: https://spark.apache.org/docs/latest/mllib-frequent-pattern-mining.html


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.