EDIT : If I had to match single worded phrases, I could first tokenize the text from the document and then calculate the cosine similarity of all the tokens with all the keywords from the
keyword_list. But the issue is that I might have single worded or multi worded keyphrases present in the
keyword_list. Even if I try to use
ngrams, how would I know what length of
ngrams to use?
I have searched and read many articles/questions regarding this but could not find a solution.
Problem Statement : I am trying to extract similar keywords/phrases from a document, based on a pre-curated list of keywords/phrases.
For example below is the list:
keyword_list = ['your work', 'ongoing operations', 'completed operations', 'your name', 'bodily injury', 'property damage',
'to the extent permitted by law', 'is required by a contract or agreement']
I also have the text I extracted from the documents using OCR. Let's say the text is as below:
text = "In light of your ongoing operations, your name is an approximation of your working models. The contract requires that the damage done to the property must be borne by both the parties, as permitted by the law."
Now I want to extract all the keywords/phrases that occur in the keyword_list. In addition to that I also want to extract similar keyphrases (by similar I mean similar in context or meaning but worded differently). So the logic/model should be able to extract the following terms:
output = ["ongoing operations", "your name", "your working", "The contract requires", "damage done to the property", "as permitted by the law"]
We can see that
ongoing operations and
your name are present in the
keyword_list and hence are extracted.
The contract requires,
damage done to the property,
as permitted by the law are also extracted because they have the same meaning/context to
is required by a contract or agreement,
to the extent permitted by law.
For the phrases matching completely (
ongoing operations and
your name), I have written a logic which uses regex to match the phrases. But for the phrases which have the same meaning/context but worded differently, I am unsure how to proceed. I think a Machine learning or Deep learning approach would be suitable here but I don't know which exact approach!
Any help is appreciated!