I'm having a hard time understanding why people use any vector they find as a candidate for a recommender system.

In my mind, a recommender system requires a space where distance represents similarity. Of course, before you can construct such a space, first you need to settle on the type of distance you want to use (euclidean, angular, or anything else). Then you need a model (assuming we are talking about ML) to map your input (it could be an image, text, or anything else) to a point in that space. One major aspect of this model is that it's aware of the type of distance we've defined. If there's no notion of the distance in the model, definitely the output of the model is not going to have the attribute of "distance means similarity".

I'm asking this question because I've seen people use any vector they find to construct a recommender system. Here's an example of using a VAE's latent vectors for recommender systems:


I've also seen people using fastText word embeddings in the same way. I understand that all these embeddings/latent vectors form clusters in their spaces with some interesting patterns. But I don't think this is enough to assume the "distance represents similarity" requirement for a recommender system.

Please let me know if I'm missing anything here.

  • $\begingroup$ I think the training process forces the vectors that do the best job to emerge. Posting this as a comment, as "it works because it works" doesn't feel like the answer you were after :-) $\endgroup$ Feb 15 at 21:21
  • $\begingroup$ @DarrenCook But the question is "does it really work?" or they are just assuming that it does? $\endgroup$
    – Mehran
    Feb 20 at 15:55
  • $\begingroup$ It is trivial to make word embeddings, and reproduce results; or simply use a freely available pre-trained model. Contextual embeddings (like BERT) beat simple word embeddings (wordvec, fasttext, etc.) beat the algorithms that came before, in just about all NLP tasks. They are not perfect, but that is what I meant by "it works". $\endgroup$ Feb 20 at 16:48

You are correct that recommender systems that map similarity to distance is useful.

Vector representations are useful because most machine learning learning tools are based on linear algebra. Vector representations encode raw data in form that amenable to machine learning.

"Any" vector representation is more useful than no vector representation. For example, one-hot encoding is often more useful than not including the feature. However, distance in one-hot encoding is not related to similarity.

What is even more useful are semantic vectors (e.g., word2vec and related techniques). Semantic vectors map contextual meaning into locations in a learned vector space. This is an example of where distance is a proxy for similarity.

  • $\begingroup$ Thanks for the answer. But I believe your very first sentence needs tweaking. "similarity to distance is useful" should be "distance to similarity is necessary". I believe if you have your data mapped into a vector space where distance does not represent similarity, that space cannot be used by a recommender system directly. $\endgroup$
    – Mehran
    Jul 25 at 15:37
  • $\begingroup$ @Mehran One-hot encoding can be directly used by an ML recommender; as Brian says here, distance is not related to similarity, yet it is superior to not using the data at all. $\endgroup$ Jul 25 at 19:12
  • $\begingroup$ I appreciate it if you could point me to (or explain) how a space without similarity <=> distance can be utilized for a recommender system. Based on my knowledge (which is not extensive by any means), spaces (like one-hot vectors) where distance does not represent similarity are absolutely useless for a recommender system. Please remember that I'm talking about the output of the ML model which means there cannot be any more mapping of the vectors (these vectors will be fed into ANN search engine). $\endgroup$
    – Mehran
    Jul 25 at 19:26
  • $\begingroup$ Simply put, whatever you will be feeding into an ANN search engine has to have the attribute of "distance represents similarity". Otherwise, the query results that you'll get from the ANN will be meaningless. ANN: Approximate Nearest Neighbor. $\endgroup$
    – Mehran
    Jul 25 at 19:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.