Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I am trying to build a recommendation system using collaborative filtering. I have the usual [user, movie, rating] information. I would like to incorporate an additional feature like 'language' or 'duration of movie'. I am not sure what techniques I could use for such a problem.
Please suggest references or packages in python/R.
Here some resources that might be helpful:
Recommenderlab - a framework and open source software for developing and testing recommendation algorithms. Corresponding R package recommenderlab.
The following blog post illustrates the use of recommenderlab package (which IMHO can be generalized for any open source recommendation engine) for building movie recommendation application, based on collaborative filtering.
Research on recommender systems - a nice webpage with resources on the topic, maintained by Recommenderlab's lead developer Michael Hahsler.
Mortar Recommendation Engine - an open source customizable recommendation engine for Hadoop and Pig, written in Python and Java. Company, sponsoring the development of this project, Mortar Data, offers general commercial cloud platform for development and hosting data science software projects, including ones based on the Mortar Recommendation Engine (development and hosting of public projects are free): http://www.mortardata.com. Mortar Data provides help in form of public Q&A forum (https://answers.mortardata.com) as well as a comprehensive tutorial on building recommendation engine using open technologies (http://help.mortardata.com/data_apps/recommendation_engine).
"Introduction to Recommender Systems" - a relevant Coursera course (MOOC), which content and description provide additional resources on the topic.
PredictionIO - an open source machine learning server software, which allows building data science applications, including recommendation systems (source code is available on GitHub). PredictionIO includes a built-in recommendation engine (http://docs.prediction.io/current/engines/itemrec/index.html) and supports a wide range of programming languages and frameworks via RESTful APIs as well as SDKs/plug-ins. PredictionIO maintains an Amazon Machine Image on AWS Marketplace for deploying applications on the AWS infrastructure.
Additional open source software projects, relevant to the topic (discovered via MLOSS website on machine learning open source software: http://www.mloss.org):
The following relevant R blog posts are also interesting:
Instead of collaborative filtering I would use the matrix factorization approach, wherein users and movies alike a represented by vectors of latent features whose dot products yield the ratings. Normally one merely selects the rank (number of features) without regard to what the features represent, and the algorithm does the rest. Like PCA, the result is not immediately interpretable but it yields good results. What you want to do is extend the movie matrix to include the additional features you mentioned and make sure that they stay fixed as the algorithm estimates the two matrices using regularizastion. The corresponding entries in the user matrix will be initialized randomly, then estimated by the matrix factorization algorithm. It's a versatile and performant approach but it takes some understanding of machine learning, or linear algebra at least.
I saw a nice ipython notebook a while back but I can't find it right now, so I'll refer you to another one which, while not as nice, still clarifies some of the maths.
