2
$\begingroup$

I have a set of 10 verses from the Bible in English. I want to detect the occurrence of any of these verses in a text. What would be the best way to go about doing this?

Note that verses of the Bible are worded differently from one translation to another. For example, the verse "Cast all your anxiety on him because he cares for you" exists in other translations as:

KJV Casting all your care upon him; for he careth for you.

ESV casting all your anxieties on him, because he cares for you.

NLT Give all your worries and cares to God, for he cares about you.

Also, people may make typos or make mistakes when writing a verse.

Would you say Semantic Search is the best approach for this? If so, do I use a pertained word embeddings model, retrain it on my set of 10 verses, and then search my text for the occurrence of these verses?

How would the search mechanism work? Do I get the word embedding sentence by sentence and compare each sentence's embedding one by one with my Bible corpus of 10 verses' embeddings?

$\endgroup$
2
$\begingroup$

One approach is using Word Mover’s Distance (WMD). WMD is an algorithm for finding the distance between texts of different lengths, where each word is represented as a word embedding vector.

The WMD distance measures the dissimilarity between two text documents as the minimum amount of distance that the embedded words of one document need to "travel" to reach the embedded words of another document.

For example:

enter image description here Source: "From Word Embeddings To Document Distances" Paper

Each sentence in the text could be compared to the 10 Bible verses. The bible verses could be ranked based on similarity.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.