0
$\begingroup$

I have constructed a natural language processing (NLP) model with the aim of identifying technology keywords within text. The model is trained on a large dataset that contains over 400,000 phrases and has been annotated with approximately 1000 technology keywords, of which only the keywords that I provided in the dataset, can be identified. The annotations within the training dataset include specific locations of technology keywords in the phrases, like for example in the below example, the technology keyword "php" is located at positions 0-3 and 43-46.

TainingData = [ ('php search upperlower case mix word string php , regex , search , pregmatch , strreplace',
  {'entities': [[0, 3, 'php'], [43, 46, 'php']]}),
 ('create access global variables groovy groovy',
  {'entities': [[31, 37, 'groovy'], [38, 44, 'groovy']]}),
 ('asp.net mvc 2.0 application fail error parameterless constructor define object asp.net , asp.netmvc , asp.netmvc2',
  {'entities': [[0, 7, 'asp.net'],
    [79, 86, 'asp.net'],
    [89, 99, 'asp.netmvc']]}),
 ('question regular servlets within gwt work dev mode work deployment tomcat java , gwt , servlets , fileupload',
  {'entities': [[74, 78, 'java']]}),
 ('display type ive create use create type postgresql database , postgresql , type , export',
  {'entities': [[40, 50, 'postgresql'], [62, 72, 'postgresql']]}),
 ('compare date specific one datetime string twig php , twig',
  {'entities': [[42, 46, 'twig'], [47, 50, 'php'], [53, 57, 'twig']]}),
 ('ie display simple js alert javascript , internetexplorer7 , parallel',
  {'entities': [[27, 37, 'javascript']]}),
 ('differences basehttpserver wsgiref.simple_server python , basehttpserver , wsgiref',
  {'entities': [[49, 55, 'python']]})
]

An alternative very simple approach that I considered is to manually search for the technology keywords within the text, by using a set of predefined keywords and checking the text in a loop. However, I am wondering of which approach would be more efficient and effective. Given the large dataset and a vast number of keywords, I wonder which of these two methods would yield the best results and would be more appropriate.

$\endgroup$

2 Answers 2

1
$\begingroup$

To add to @noe answer, you will face more issues when working with real data. Here are some examples:

  1. You might find sticky words. Ex: python,django,fastapi.
  2. You might find an alternative word. Ex: Python3.7.
  3. Sentences might be longer in real life than your training data.

It will depend on how you prepare the sentences, the side of data you're extracting skills from, and your resources. For instance, do you have GPU or CPU, RAM ... etc.

$\endgroup$
1
  • $\begingroup$ thank you for the facts. really appriciate your answer. $\endgroup$ Commented Jan 12, 2023 at 3:15
1
$\begingroup$

Any a priori answer would be pure speculation.

To answer such a question, I recommend you prepare a test set with sentences and annotated keywords, and test both your NLP model and the simple list-based approach.

The test must be representative of the kind of data that the system will face when actually used in production. Of course, this test set must not overlap with the NLP model training data.

$\endgroup$
1
  • $\begingroup$ I have already done it. thank you for the suggestion anyway. accuracy is high in nlp model and time is less in the keyword search. $\endgroup$ Commented Jan 12, 2023 at 3:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.