I have few sentences like below in my project(around 25000),

sentence1 = 'Must be able to multi-task in a fast-paced, deadline-driven environment'
sentence2 = 'Strong organisational skills and proactive approach'
sentence3 = 'in this role you will design develop and revise application simulations in alignment with product implementation timelines'
sentence4= 'ensures appropriate cross-referencing between documents within and across qms chapters/topics'

From these sentences, I am trying to determine what functionality is being expected from the person or like what person has to do. I need to identify those keywords. I can use any ML techniques or any other method to determine the keywords,

for example in

sentence1 = [multi-task, fast paced, deadline-driven environment]
senetence3 = [design develop and revise application simulations]

I tried to identify those words with help of POS but that wasn't very useful. Are there any algorithms or ideas or methods that can detect such keywords ?

  • $\begingroup$ Please clarify the question. What are the keywords you’re trying to extract exactly? How does “able to multi-task” differ from “in this role”? I mean I understand that “in this role” explains nothing about what is expected from a candidate, but how should the algorithm know this? $\endgroup$ – Paul Aug 10 '19 at 12:34

Since you don't seem to have any annotated data, the best you can do is probably this:

  1. Optional first step: remove stop words (there are many such lists available, for example https://pythonspot.com/nltk-stop-words/)
  2. Calculate the Inverse Document Frequency for every word in the vocabulary. This is intended to measure the importance of a word based on how often it's used, in the sense that a word used in fewer sentences makes it more important for these sentences.
  3. Calculate TF-IDF for every word in a sentence, then rank the words by their TF-IDF weights. In general the top ones are supposed to be the most relevant. Optionally you can decide to select the top N words as keywords.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.