32 votes

Meaning of latent features?

At the expense of over-simplication, latent features are 'hidden' features to distinguish them from observed features. Latent features are computed from observed features using matrix factorization. ...
user avatar
23 votes

Item based and user based recommendation difference in Mahout

Item Based Algorithm ...
user avatar
  • 1,025
17 votes

Recommending movies with additional features using collaborative filtering

Here some resources that might be helpful: Recommenderlab - a framework and open source software for developing and testing recommendation algorithms. Corresponding R package recommenderlab. The ...
user avatar
15 votes
Accepted

Does click frequency account for relevance?

Depends on the user's intent, for starters. Users normally only view the first set of links, which means that unless the link is viewable, it's not getting clicks; meaning you'd have to be positive ...
user avatar
  • 1,892
14 votes
Accepted

Can I use cosine similarity as a distance metric in a KNN algorithm

Short answer: Cosine distance is not the overall best performing distance metric out there Although similarity measures are often expressed using a distance metric, it is in fact a more flexible ...
user avatar
  • 276
12 votes
Accepted

Item based and user based recommendation difference in Mahout

You are correct that both models work on the same data without any problem. Both items operate on a matrix of user-item ratings. In the user-based approach the algorithm produces a rating for an item <...
user avatar
  • 286
11 votes
Accepted

Create most "average" cosine similarity observation

You are doing the correct thing. Technically, this averaging leads to computing the centroid in the Euclidean space of a set of N points. The centroid works pretty well with cosine similarities (...
user avatar
  • 1,516
10 votes

Spark ALS: recommending for new users

Lots of questions here. First, for a truly new user with no data, there is no way to use a recommender model. If you have literally no information on the user, the only thing you can do is provide ...
user avatar
  • 6,415
9 votes

Meaning of latent features?

Suppose you have (MxN) sparse matrix, where M -- stands for number of users who gave recommendations, and ...
user avatar
9 votes

Preference Matching Algorithm

My first suggestion would be to somehow map the non-quantifiable attributes to quantities with the help of suitable mapping functions. Otherwise, simply leave them out. Secondly, I don't think that ...
user avatar
  • 1,516
8 votes
Accepted

Difference between using RMSE and nDCG to evaluate Recommender Systems

nDCG is used to evaluate a golden ranked list (typically human judged) against your output ranked list. The more is the correlation between the two ranked lists, i.e. the more similar are the ranks of ...
user avatar
  • 1,516
8 votes
Accepted

Benchmark datasets for collaborative filtering

The obvious answer would be the Netflix prize dataset, there is a lot of research into it and most CF algorithms have known scores in it. There are other available datasets that are usually used as ...
user avatar
8 votes
Accepted

Do recommendation systems necessarily use machine learning algorithms?

There's nothing about a recommendation system that absolutely necessitates some kind of machine learning. Indeed, I've seen decision systems in use that were essentially just someone's idea about what ...
user avatar
  • 268
7 votes
Accepted

How should one deal with implicit data in recommendation

Your system isn't just trained on items that are recommended right? if so you have a big feedback loop here. You want to learn from all clicks/views, I hope. You suggest that not-looking at an item ...
user avatar
  • 6,415
7 votes

Does click frequency account for relevance?

For my part I can say that I use click frequency on i.e. eCommerce products. When you combine it with the days of the year it can even bring you great suggestions. i.e.: We have historical data from ...
user avatar
  • 579
7 votes

Preference Matching Algorithm

As suggested, going wild. First of all, correct me if I’m wrong: Just a few features exist for each unique product; There is no ultimate features list, and clients are able to add new features to ...
user avatar
  • 1,109
7 votes
Accepted

How do you calculate how dense or sparse a dataset is?

It's actually defined on the first page: ... sparsity level (ratio of observed to total ratings) ... In other words, the fraction of the user/item rating matrix that is not empty. Remember that ...
user avatar
  • 10.4k
7 votes

spark item similarity recommendation

For your recommendation engine, if you've chosen to go by item similarity approach, then you can use Spark's RowMatrix datatype to achieve this task. Item similarity approach is just about creating ...
user avatar
7 votes
Accepted

How is the cross-product transformation defined for binary features?

Let's do this in the opposite order of how you asked. i.e. first: How can I think of the cross-product transformation in general? For me a cross-product comes from linear algebra, and it is a ...
user avatar
  • 472
7 votes

What is difference between Nearest Neighbor and KNN?

Not really sure about it, but KNN means K-Nearest Neighbors to me, so both are the same. The K just corresponds to the number of nearest neighbours you take into account when classifying. Maybe what ...
user avatar
  • 561
6 votes

What kind of data is not appropriate using CF to do recommendation?

The key is establishing a proper validation metric. I notice you talk about how you tried different recommendation algorithms, but at the end of the day you evaluated them all with RMSE. But there's ...
user avatar
6 votes

Clustering users based on buying behaviour

Big Picture: First of all, the feature set in your data is pretty sparse and uninteresting, so you should not expect to gain much traction from this problem. Use your human mind to think about the ...
user avatar
  • 6,668
6 votes
Accepted

Recommender system based on purchase history, not ratings

You could use Content based filtering but then you have to intelligently pre process the data to extract all the contents of the products. Also, that might lead to leaving a some features, This ...
user avatar
6 votes

Factorization Machine - prevent over fitting

Here are some excerpts from the original paper that I think are key to understanding the question: Instead of using an own model parameter for each interaction, the FM models the interaction by ...
user avatar
  • 313
6 votes

Why is deep learning used in recommender systems?

"Recommender Systems" is a very broad area and can be approached from different optics: latent variable models, graph models, etc. "Deep learning" is an umbrella term for gradient-...
user avatar
  • 15.4k
6 votes

How to evaluate when recommender systems are influencing behavior?

This is a case for A/B testing: Two versions of the website are prepared, one offering the coupon and another not offering it. Users are randomly split into two groups, some of them getting the coupon ...
user avatar
  • 21.8k
6 votes

How to evaluate when recommender systems are influencing behavior?

As Erwan has pointed out, the straightforward way to tackle this problem is using A/B testing, so I'll point out another technique to solve the issue, which is counterfactual evaluation. In ...
user avatar
  • 489
5 votes

Meaning of latent features?

Another example, consider the case of users to movie rating matrix like the Netflix setup. This will be a huge sparse matrix which is difficult to process. Note that each user will have a specific ...
user avatar
  • 51
5 votes

Does click frequency account for relevance?

Is it valid to use click frequency, then yes. Is it valid to use only the click frequency, then probably no. Search relevance is much more complicated than just one metric. There are entire books on ...
user avatar
  • 1,102
5 votes

Business exception reporting

Try exploring the rich field of "Anomaly Detection in Time Series". Control charts and CUSUMs (or cumulative sum control charts) might help you. Simple Bullet Graphs might be all you need. Based on ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible