0
$\begingroup$

In text embeddings, cosine similarity is often used to find texts similar to a search query. However, I don't want to find a text that is overall similar, but similar with regards to a specific concept (which I can also embed).

Example: Let's say you have many movie reviews that you have embedded. You choose one review and want to search for similar ones with regard to the cinematography only.

More formally stated, my problem is the following: Let $x_1, \dots, x_n \in \mathbb{R}^d$ vector embeddings of texts $t_1, \dots, t_n$. Further, let $q \in \mathbb{R}^d$ be the embedding of a search query. I want to rank the texts by similarity with the search query with regards to a specific concept/aspect with embedding $k \in \mathbb{R}^d$.

I thought about using the embedding $k$ of the concept, like "cinematography" and then projecting the text embeddings onto that direction $$S_C \propto \langle P_k q, P_k x_i\rangle$$

but that does not make sense as cosine similarity looks at the angle.

One could project just one of the text embeddings $$S_C \propto \langle P_k q, x_i\rangle = S_c(k, q) \cdot S_c(k, x_i)$$ but this amounts to multiplying the cosine similarities of the concept with both text embeddings.

I did not find any research literature on this specific problem. Is there a way to align the cosine similarity to a specific concept/context?

$\endgroup$

2 Answers 2

1
$\begingroup$

You can generate custom embeddings for your corpus/dataset and then calculate the cosine similarity. When you generate your own word embeddings for your dataset, words with similar meaning will be close to each other in vector space. So even if two words don't have any letter in common but are in same context would have some value of cosine similarity. Below is the link of a tensorflow tutorial of generating custom embeddings - https://www.tensorflow.org/text/guide/word_embeddings

$\endgroup$
3
  • $\begingroup$ I have embeddings for many texts. Those use similar meaning, not spelling, but I want to find similarity wrt some specific aspect of the texts. Maybe this is not possible with training something, but I thought it might. $\endgroup$
    – McLawrence
    Jan 23, 2023 at 13:36
  • $\begingroup$ @McLawrence Please tell more about which 'specific aspect' of texts, you want to find similarity of. $\endgroup$
    – YadneshD
    Jan 24, 2023 at 10:38
  • $\begingroup$ Like in the example in the original post. Another example would be something like this: You have restaurant reviews and embed them (not per word, but one vector for the entire review). Now I don't want to find generally similar restaurant reviews, but for example only with regard to the atmosphere in the restaurant, not the food. $\endgroup$
    – McLawrence
    Jan 24, 2023 at 13:13
0
$\begingroup$

Could you treat this like a classification problem? As in, assume that the concepts have subclasses and consider texts to be similar if they are in the same class?

It's hard to pin down specific concepts in an embedding space. The dimensions are effectively meaningless, but you can get some information by comparing embeddings with other embeddings in the space. The Euclidean properties of an embedding space do sometimes hold information about specific concepts; like the classic example in word embeddings where ('King' - 'Man' + 'Woman') ≈ 'Queen' suggests that the vector ('King' - 'Queen') is something like a masculine/feminine axis. But this is oversimplified, because the words King and Queen aren't used in identical contexts, so the difference in their meaning isn't just a single concept.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.