16
votes
Accepted
Can I use cosine similarity as a distance metric in a KNN algorithm
Short answer: Cosine distance is not the overall best performing distance metric out there
Although similarity measures are often expressed using a distance metric, it is in fact a more flexible ...
- 296
10
votes
Spark ALS: recommending for new users
Lots of questions here. First, for a truly new user with no data, there is no way to use a recommender model. If you have literally no information on the user, the only thing you can do is provide ...
- 6,595
9
votes
Meaning of latent features?
Suppose you have (MxN) sparse matrix, where M -- stands for number of users who gave recommendations, and ...
- 236
8
votes
Accepted
Benchmark datasets for collaborative filtering
The obvious answer would be the Netflix prize dataset, there is a lot of research into it and most CF algorithms have known scores in it.
There are other available datasets that are usually used as ...
- 311
8
votes
Accepted
Do recommendation systems necessarily use machine learning algorithms?
There's nothing about a recommendation system that absolutely necessitates some kind of machine learning. Indeed, I've seen decision systems in use that were essentially just someone's idea about what ...
- 268
8
votes
Accepted
How is the cross-product transformation defined for binary features?
Let's do this in the opposite order of how you asked. i.e. first:
How can I think of the cross-product transformation in general?
For me a cross-product comes from linear algebra, and it is a ...
- 482
7
votes
Accepted
How do you calculate how dense or sparse a dataset is?
It's actually defined on the first page:
... sparsity level (ratio of observed to total ratings) ...
In other words, the fraction of the user/item rating matrix that is not empty. Remember that ...
- 10.5k
7
votes
spark item similarity recommendation
For your recommendation engine, if you've chosen to go by item similarity approach, then you can use Spark's RowMatrix datatype to achieve this task.
Item similarity approach is just about creating ...
- 395
7
votes
What is difference between Nearest Neighbor and KNN?
Not really sure about it, but KNN means K-Nearest Neighbors to me, so both are the same. The K just corresponds to the number of nearest neighbours you take into account when classifying.
Maybe what ...
- 626
6
votes
Clustering users based on buying behaviour
Big Picture:
First of all, the feature set in your data is pretty sparse and uninteresting, so you should not expect to gain much traction from this problem. Use your human mind to think about the ...
- 6,798
6
votes
Accepted
Recommender system based on purchase history, not ratings
You could use Content based filtering but then you have to intelligently pre process the data to extract all the contents of the products. Also, that might lead to leaving a some features, This ...
- 146
6
votes
Factorization Machine - prevent over fitting
Here are some excerpts from the original paper that I think are key to understanding the question:
Instead of using an own model parameter for each interaction,
the FM models the interaction by ...
- 313
6
votes
Why is deep learning used in recommender systems?
"Recommender Systems" is a very broad area and can be approached from different optics: latent variable models, graph models, etc.
"Deep learning" is an umbrella term for gradient-...
- 23.8k
6
votes
precision@k and recall@k
A quick answer.
Note: Checking the references I could access fully, there are no discrepancies between the definitions as long as the terms are translated correctly.
Some definitions:
Relevant items: ...
- 2,301
6
votes
How to evaluate when recommender systems are influencing behavior?
This is a case for A/B testing:
Two versions of the website are prepared, one offering the coupon and another not offering it. Users are randomly split into two groups, some of them getting the coupon ...
- 25k
6
votes
How to evaluate when recommender systems are influencing behavior?
As Erwan has pointed out, the straightforward way to tackle this problem is using A/B testing, so I'll point out another technique to solve the issue, which is counterfactual evaluation.
In ...
- 536
6
votes
How to build a personalized recommendation system using real life scenario data?
100k apps, 300k users - quite the task.
Modern ranking systems typically consist of 3 phases.
Recall (Generate candidates)
For each user, reduce the number of applications you could recommend them ...
- 435
5
votes
Meaning of latent features?
Another example, consider the case of users to movie rating matrix like the Netflix setup. This will be a huge sparse matrix which is difficult to process.
Note that each user will have a specific ...
- 51
5
votes
Item Based Collaborative Filtering with No Ratings
There might be different ways to do that, like considering implicit ratings like views or clicks.
But basically, you can consider a rating of 1.0 for each user-item pair you have.
This way, your ...
- 545
5
votes
Accepted
Which supervised learning algorithms are available for matching?
You can try to frame this problem as a recommender systems situation. Where you have your users (prospective students) and items (alumni) and want to recommend to the users one item.
It's not a ...
- 311
5
votes
Recommender system based on purchase history, not ratings
there are no product ratings available, thus collaborative filtering
is not an option
Wrong. You can do collaborative filtering with holdings. Just use the numbers/duration of holdings instead of ...
- 1,307
5
votes
Calculate similarity on boolean data
You should look at the Jaccard Index, is the de facto similarity between set of items, where the sets are represented using a boolean vector. In this boolean vector each coordinate represents an item, ...
- 2,226
5
votes
Mean Average Precision python code
This library called Metrics provides most of metrics for Machine Learning including MAP for Recommendation systems. If you only interested in metrics for recommendation systems, perhaps you can see ...
- 4,207
5
votes
How to use ndcg metric for binary relevance
The nDCG depends on the relevance of each document as you can see on the Wikipedia definition. I guess you could use 0 and 1 as relevance scores, but then all relevant documents would have the same ...
- 51
4
votes
How to split train/test in recommender systems
Leave-one-out cross validation is probably the most straight-forward way to address this. If you happen to be using a model that requires a lot of time to train, ...
- 6,798
4
votes
How can conclusions be drawn from recommendation systems evaluation?
"Good", I think, is based on the state of the art at the moment. So I would look at respected models from industry leaders and use their reported accuracies as a base line for what is "good": since ...
- 567
4
votes
how to evaluate top n recommendation system with movie lens dataset?
For various metrics feel free to look at various benchmarking libraries including MyMediaLite and LibRec. If you are doing a TOP N approach, then the way to evaluate this using a Movielens system is ...
- 141
4
votes
Accepted
Interpretation of an SVD for recommender systems
First of all, note that the dot product between two movies/users is by definition the correlation between them. So your intuition for treating it as a similarity measure is not wrong.
Now, applying ...
- 241
4
votes
Is there an overview over recommender system architectures?
A great source is the personal page of Xavier Amatriain (former Head of Engineering at Netflix and Quora). There are several links at publications detailing the system and architecture of recommender ...
- 2,226
4
votes
Mean Average Precision python code
You can use the ml_metrics library. For install this library use:
pip install ml_metrics
import ml_metrics
ml_metrics.mapk(actual, predicted, k)
- 41
Only top scored, non community-wiki answers of a minimum length are eligible