2
$\begingroup$

I have a list of words and phrases (~3k items). What are my options to extract them from documents (~3M of job descriptions) with NLP? I do not have labeled data.

For example my list of words and phrases look like,

Leadership
Microsoft Office
AWS
.
.
.
Python Programing Language

The result I am looking for is a matrix(3K x 3M) with binary values inside.

Doc # Leadership Microsoft Office AWS ... Python Programing Language
1 1 0 0 1
. . . . .
3M 0 1 1 1
  1. Regex - This is the most straightforward solution comes my mind. However, this solutions is not robust and cannot capture different word/phrase forms. For example, people might write MS Office instead of Microsoft Office. Similarly, people might write Amazon Web Service rather than AWS.

  2. Is there a solution to utilize a Large Language Model such as BERT?

  3. If I create a labeled data using, for example, AWS Ground Truth, is there a way to utilize the results to build a model and extract the list of words/phrases?

$\endgroup$

2 Answers 2

1
$\begingroup$

It depends on the context imho: where does the list of words/phrases come from? Did some expert compile it thinking about every word carefully, or is it just intended as some rough indication of the actual context to be tagged?

Anyway from your description the output must use these particular phrases as columns, so it wouldn't make sense to use embeddings like BERT since the output would not really match the phrases themselves.

I don't see any point in a supervised model: what would be the target variable? All the phrases?? This would probably result in a very complex method for a very simple problem.

It might be disappointing but sometimes the simple answer is the appropriate one: direct search for these phrases, possibly after lemmatization to cover lexicographic variants.

$\endgroup$
2
  • $\begingroup$ You got me thinking, maybe use the regex output as the labeled input data and train a multi-class multi-label classifier using BERT? That could scale and possibly capture some variations such as MS Office and Microsoft Office? $\endgroup$
    – E.K.
    Commented Mar 17, 2023 at 1:55
  • $\begingroup$ @E.K. why not, but again it depends how the input list was created: if it contains both 'MS Office' and 'Microsoft Office', the output columns may contain duplicate information and probably inconsistencies. Keep in mind that there will likely be ambiguous terms or acronyms, so the question of considering the terms strictly or interpreting them broadly can give quite different results. $\endgroup$
    – Erwan
    Commented Mar 17, 2023 at 11:25
0
$\begingroup$

You have a list of keyphrases that you need to extract from documents. You could use NER for this purpose. But looking at the size of the keyphrases (3000) it will be a difficult task because you would have to first annotate the keyphrases in the documents. After that you can train a NER model to make it learn to look for those phrases and extract them.

There are many NER models out there. Start with the SpaCy library first as it gives a simple but effective framework for NER. You can try multiple BERT based models using the SpaCy library. You can use any of the NLP models available on HuggingFace inside SpaCy for NER.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.