1
$\begingroup$

I'm trying to similar research abstracts, so I'm using word embeddings to convert words into 1x768 vectors, so overall turning abstracts into embeddings with shape (#ofwords, 768). Cosine similarity between two abstracts returns a matrix (#ofwords1, #ofwords2), which I then sum up to get an overall score. What I'm wondering is if this summing up of all the values in a cosine similarity matrix is really a good way to determine overall similarity between two different texts? Is there a better, or less computationally expensive way to do this?

$\endgroup$
1
  • $\begingroup$ It's not very good because you're comparing every word with every other word, so your matrix is probably full of low scores which irrelevant and this is a potential bias. I'm not expert with embeddings but I think you could use document embeddings and calculate similarity between two document vectors. Otherwise you could try representing documents with traditional TFIDF vectors. $\endgroup$
    – Erwan
    Aug 27, 2020 at 14:48

2 Answers 2

0
$\begingroup$

A computationally simpler way is to compare documents is by calculating the cosine similarity of the average of the words embeddings for each document. The average of the word embeddings for a given document is one way to summarize the document.

$\endgroup$
0
$\begingroup$

A similar but more advanced approach would be BERTScore. It also computes pairwise cosine similarity between (BERT) embeddings but uses greedy matching by only accounting for similarity between the closest tokens: enter image description here (based on Figure 1 from the BERTScore paper)

However, it should be noted that BERTScore is designed to be used for paragraphs and not documents.

Another more traditional approach would be doc2vec.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.