I have data in the following format.

Customer Product_Bought Date_of_purchase
C1       P1             D1
C1       P2             D2
C2       P3             D3 
C2       P1             D2 
C2       P2             D4
C3       P1             D5
C3       P3             D5 

I am trying to build a recommender system to recommend next product to users. I just have this purchase history . What should i use in python to recommend products . Also , I have read other answers which are possible duplicates and i didn't find anything substantial


This is a typical recommendation system. Just to recap, the three most popular ones are:

  • Collaborative models use only collaborative information – implicit or explicit interactions of users with products. They don’t use any information on the actual product attributes (like brand, category, color, etc.). These models can achieve high precision with little data, but they can’t handle unknown users or items (the cold start problem).

  • Content-based models work purely on the meta data about items or users – entirely ignoring interactions between users and items. These models approach recommendations very differently than collaborative models and can address the cold start problem. But they require much more data for training.

  • Hybrid Recommenders: combine both approaches and overcome a lot of the challenges of each individual approach.

I suggest starting with the most straightforward method Collaborative filtering. This method is used when one doesn't have or so to say doesn't want to include any features about the products – or the features aren't informative. This mean you may start with just the purchase history. The state-of-the-art collaborative filtering algorithms are implemented with a matrix factorization (MF). Here we estimate two latent (unobserved) matrix representations, which, when multiplied by each other, will reproduce the matrix of interactions for each item and user the model saw during training. Here is the schematic view of MF for Movie Recommendations to Users:

enter image description here

You may want to have a look at LightFM , a very popular python recommendation library that is suited for small- to middle-sized recommender projects. This notebook just a super quick jump into this method. Please note that there, you have to have the recsys.py stored locally, since there are multiple functions they use in the notebook. If you run into issues making this work let me know.

Last but not least, I see you have date in your data. You do not need date. Do not get confused by the concept of ignoring it, we do not. What people usually do is that, they estimate this big interaction matrix (i.e. recommendations) using MF once per day overnight, so whenever a new purchase is made by a user, that user's purchase vector gets updates and MF will get updated for the next day. And they cache the results for quick recommendation during the day. Unless more faster response is needed (anything faster than one day), either MF wouldn't be the best (see the Bonus below), or it might be possible to recalculate it a few times per day to update the recommendation matrix. This is a business decision. Imagine how fast users buy from a platform anyway. Maybe once every now or then or once per day. Wonder how buys a few times per day.

Bonus (Advanced): Deep Matrix Factorization I would strongly suggest starting off with MF. However be informed that MF is rather old, and surely there are recent advancements. MF is a linear method, meaning that if there are complicated non-linear interactions going on in the data set, a simple dot product may not be able to handle it well. Given the recent success of deep learning in complicated non-linear computer vision and natural language processing tasks, it is natural to want to find a way to incorporate it into matrix factorization as well. A way to do this is called deep matrix factorization and involves the replacement of the dot product with a neural network that is trained jointly with the factors. This makes the model more powerful because a neural network can model important non-linear combinations of factors to make better predictions. A comparison of the two methods:

enter image description here

In traditional MF, the prediction is the simple dot product between the factors for each of the dimensions. In contrast, in deep matrix factorization the factors for both are concatenated together and used as the input to a neural network whose output is the prediction. The parameters in the neural network are then trained jointly with the factors to produce a sophisticated non-linear model for matrix factorization.

References that I've used to compile the answer (and I personally implemented mine starting from these links): article in towardsdatascience, blog-post, article with a detailed overview and credit for the images goes to an article in Deep matrix factorization using Apache MXNet in O'Reilly (for DMF implementation I suggest starting with the O'Reilly article).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.