What are good ways to find for single sentence (query) the most similiar document (text). I asked myself if word vectors (weighted average of the documents) are suitable to map a single sentence to a whole document?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Doc2Vec is on possible approach. With this, model learns to "cluster" similar sentences together.
Most simplistic approach is to aggregate word vectors but that ignores order of words. Details on few of the approaches :
https://towardsdatascience.com/sentence-embedding-3053db22ea77 https://medium.com/explorations-in-language-and-learning/how-to-obtain-sentence-vectors-2a6d88bd3c8b
answered Jan 18, 2019 at 17:48
-
$\begingroup$ I thought of that too. But I wonder if this is suitable to map a single sentence to a text consisting with let's say 50 sentences? $\endgroup$– TidoJan 18, 2019 at 17:50
-
$\begingroup$ That is somewhat specific to problem. For example, this works very well for mapping sentences to WikiPedia articles (due to diversity of topics, separation is easier). It might not work as well if all documents are from very similar domains. $\endgroup$ Jan 18, 2019 at 17:53