0
$\begingroup$

I am trying to build a resume parser which can extract details such as Name, Address, Education details (degree name, college name, university name, course duration), Experience details (designation, company name, company location, work duration) from any kind of resume.

I tried to train a custom ner model using spacy. For that I created annotations from resumes which have entities as follows:

Degree -> Degree name, College -> College name, University -> University name, Degree_date -> Degree date.

Similarly created entities for experience too.

So i extracted text from the resume, for preprocessing I have done:

  • Removed new lines, extra spaces, html tags.
  • Then removed special symbols such as bullet symbols etc.
  • Also encoded to ascii format so that some other kind of symbols will be removed

The resultant text is used to annotate the entities.

Then I trained the model but it is not working as expected. It cannot extract all the details and sometimes the entities are wrongly detected.

Rule based extractor cannot be considered.

I want to know:

  1. Why my custom ner model is not extracting properly and not able to extract the text in the order as in resume.
  2. Any other possibility is there?
  3. Is it possible to use bert for this? If so, how should i structure the annotaions or in what format should i create the dataset for training bert?
  4. If there is any other approach, please specify that too?

Any help or suggestion will be greatly appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Try using the spacy-transformers for NER.

You can even Fine-Tune it as per the project requirements. Refer to this link: https://towardsdatascience.com/how-to-fine-tune-bert-transformer-with-spacy-3-6a90bfe57647

Improve this answer
$\endgroup$
3
  • $\begingroup$ how much data will be needed to train the transformer? $\endgroup$
    – SRJ577
    Nov 11, 2021 at 13:56
  • $\begingroup$ If you are directly using the spacy-transformers package, no training is required. However, even if you want to fine-tune it, the model can be fine-tuned with fewer amounts of data. Refer to the link mentioned for more details. $\endgroup$
    – Shrinidhi M
    Nov 12, 2021 at 7:35
  • $\begingroup$ I found this article on adding custom entities to NER in Spacy: Good accuracy is achieved with 200–1000 training data. Have a look: levelup.gitconnected.com/… $\endgroup$
    – Shrinidhi M
    Nov 24, 2021 at 13:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.