0
$\begingroup$

Referring to the LightFM model from paper Metadata Embeddings for User and Item Cold-start Recommendations, the model tries to learn $d$-dimensional user and item feature embeddings $e_f^U$ and $e_f^I$ for each feature $f$ ($U$ is the set of users, $I$ is the set of items).

The latent representation of user $u$ is given by the sum of its features' latent vectors: $$ \mathbf{q_u} = \sum_{j\in{f_u}}\mathbf{e_j^U} $$

The same holds for item $i$: $$ \mathbf{p_i} = \sum_{j\in{f_i}}\mathbf{e_j^I} $$

Does it really make sense to sum the latent embeddings to represent a set of features (user or item)?

$\endgroup$
0
$\begingroup$

Thinking of a sum as an average, this model sounds reminiscent of continuous bag of words (CBOW) word embeddings (i.e. word2vec). In that context, the words in a sentence are used to predict a missing word using the average of embedded vectors (see this question). This method works pretty well for words, so it might make sense that you could extend it to other types of embeddings like user/item compatibility.

The average makes some intuitive sense when you think of comparisons. To place a user/item in embedding space, you can average the feature embeddings. If a user/item has different but similar features their average should put the user/item in a similar region of embedding space. A sum should work the same if the feature number is fixed.

An issue with a sum would arise if feature number isn't fixed. Users/items could appear different because their embedding vector is a different length, even if it has the same direction. Normalizing by feature number would fix this.

Another possible issue with using a sum rather than an average is that the user/item embedded vectors aren't comparable to the feature embedded vectors. This doesn't seem to matter for the author, because they don't try to compare the user/item vectors to the feature vectors directly.

TL;DR Yes, a sum might make sense to represent a set of features, but an average might be better.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.