Web Framework Built for Recommendations

Question

I'm wondering if there is a web framework well suited for placing recommendations on content.

In most cases, a data scientist goes through after the fact and builds (or uses) a completely different tool to create recommendations. This involves analyzing traffic logs, a history of shopping cart data, ratings, and so forth. It usually comes from multiples sources (the web server, the application's database, Google Analytics, etc) and then has to be cleaned up and processed, THEN delivered back to the application in way it understands.

Is there a web framework on the market which handles collecting this data up front, as to minimize the retrospective data wrangling?

Answer 1

I haven't seen anything like that and very much doubt that such frameworks exist, at least, as complete frameworks. The reason for this is IMHO the fact that data transformation and cleaning is very domain- and project-specific. Having said that, there are multiple tools that can help with these activities in terms of partial automation and integration with and between existing statistical and Web frameworks.

For example, for Python, the use of data manipulation library pandas as well as machine learning library scikit-learn can be easily integrated with Web frameworks (especially Python-based, but not necessarily), as these libraries are also Python-based. These and other Python data science tools that might be of interest can be found here: http://pydata.org/downloads. Specifically, for cleaning and pre-processing tasks, which you asked about, pandas seem to be the first tool to explore. Again, for Python, the following discussion on StackOverflow on methods and approaches might be helpful: https://stackoverflow.com/q/14262433/2872891.

Consider an example of another platform. The use of pandas for data transformation and cleaning is rather low-level. The platform that I like very much and currently use as the platform of choice for data science tasks is R. Rich ecosystem of R packages especially shines in the area of data transformation and cleaning. This is because, in addition to very flexible low-level methods of performing these tasks, there are some R packages, which take a higher-level approach to the problem, which may potentially improve developer's productivity and decrease the amount of defects. In particular, I'm talking about two packages, which I find very promising: editrules and deducorrect. You can find more detailed information about these and other R packages for data transformation and cleaning in my another answer here on Data Science StackExchange (paper that I reference in the last link there could be especially useful, as it presents an approach to data transformation and cleaning that is generic enough, so that could be used as a framework for this on any decent platform): https://datascience.stackexchange.com/a/722/2452.

UPDATE: On the topic of recommender systems and their integration with data wrangling tools and Web frameworks, you may find my other answer here on DS SE useful: https://datascience.stackexchange.com/a/836/2452.