2
$\begingroup$

This question is related to "How to grow a list of related words based on initial keywords?"

In the previous question they attempt to get similar words to a given word. However, I am interested in knowing the possibility of doing it to sentences.

As I'm not familiar with this area, my questions are:

Is there a way to do this with sentences (without just considering words)? What tools are available for it?

$\endgroup$
4
  • 1
    $\begingroup$ Welcome to the site! Use a sentence embedding (numerical representation) then perform similarity search. $\endgroup$ – Emre Aug 24 '17 at 6:08
  • $\begingroup$ Many thanks! Without developing these models from scratch, are there any available tools I can utilise? $\endgroup$ – Smith Volka Aug 24 '17 at 6:15
  • $\begingroup$ The ones I just linked to... $\endgroup$ – Emre Aug 24 '17 at 6:33
  • $\begingroup$ I am sorry as I am new to this field. But I don't see tools in those sites. Except some research papers links $\endgroup$ – Smith Volka Aug 25 '17 at 1:34
3
$\begingroup$

Word Mover’s Distance (WMD) is an algorithm for finding the distance between pairs of strings. It is based on word embeddings (e.g., word2vec) which encode the semantic meaning of words into dense vectors.

The WMD distance measures the dissimilarity between two text documents as the minimum amount of distance that the embedded words of one document need to "travel" to reach the embedded words of another document.

For example:

enter image description here Source: "From Word Embeddings To Document Distances" Paper

The gensim package has a WMD implementation.

$\endgroup$
0
2
$\begingroup$

Using the python package Fuzzy Wuzzy is also useful

It uses Levenshtein distance from the python-Levenshtein package, and gives you different options for re-arranging or using word tokens.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.