1
$\begingroup$

Assume the task here is extracting important facts for resume like a candidate skills and his education etc.., Here is resume is parsed text from pdf or docx resume.

First, I'll obtain skills and education data from various online websites, job portals etc.., the obtaining data will be two text files and in each file every row represent a skills or a university name like below,

skills.txt 
___________
c 
python 
java
node js


education.txt 
___________
massachusetts institute of Technology 
harvard university

I want to know if these be enough to be able to train a named entity recognition model to recognize skills and education for raw resume text. The data I have is not sentences but just entities. I've read somewhere that we require some context along with the entity for NER model to learn better. Like this example below,

skills.txt 
___________
c is used at facebook
python is my favorite programming language

If I use my collected data education.txt to train the modeel and BIO annotate them then it will be like below, it won't have O-Other token.

massachusetts B-EDU
institute I-EDU
of I-EDU
Technology I-EDU

harvard B-EDU
university I-EDU

indian B-EDU
institute I-EDU
of I-EDU
technology I-EDU

But I don't know how to access such data for my resume-extraction problem. How do I proceed further? How to build an effective NER model for my resume facts identification domain-specific task? Any inputs/suggestions would really help.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

If your data always looks like this, there is little reason to use sequence labeling: every token belongs to an entity, so it's just a matter of correctly separating the entities and classifying them. But since the entities are already separated by line breaks, there's no need to train a model to separate them. So in the end you just have to classify the entities by category, and this doesn't require sequence labeling. But even for that, from your example it looks like the skills vs. education entities are already separated, so in the end I'm not sure what you want the model to learn?

Improve this answer
$\endgroup$
6
  • $\begingroup$ Erwan - "My data is not like above ..my original data is parsed resume text"... I'm really sorry for the confusion. I think I haven't clearly articulated my problem. I hope you don't mind. I have updated my question a little better I really hope it's clear now. $\endgroup$
    – user_12
    Commented May 14, 2020 at 13:21
  • $\begingroup$ @user_12 I'm still a bit confused: do you mean that in your example the first two files 'skills.txt' and 'education.txt' are the results that you are trying to obtain, and the second file 'skills.txt' file ('c is used at facebook...") is your input text? That would be a more standard case for sequence labeling indeed. $\endgroup$
    – Erwan
    Commented May 14, 2020 at 14:39
  • $\begingroup$ Actually my input text is just like in first two files skills.txt and education.txt and I want to train the model using that. But I'm not sure if it works or not because it doesn't have any context information around it. So, my question, is it possible to train a resume extraction NER model using the combination of those two files as input and train it to learn entities then inference it on raw resume text. $\endgroup$
    – user_12
    Commented May 14, 2020 at 16:22
  • 1
    $\begingroup$ @user_12 ok in this case my answer applies: technically you could train such a model, it doesn't matter that there's no O label. However you wouldn't really be using the sequence labeling capabilities of the algorithm since your data contains very little sequential information. For instance the order of the skills "c, python" doesn't matter, it could as well be "python, c", so the algorithm cannot pick any information from the previous word. So at best that would be a waste of computing time compared to just extracting each skill one by line (plus it's likely not to perform as well). $\endgroup$
    – Erwan
    Commented May 14, 2020 at 16:34
  • $\begingroup$ Is there any other way to collect or prepare data for my use case. My project is extracting relevant information from resumes like, skills, name, email, educations, projects, work experiences etc.., Do I have to manually annotate each resume in order it to perform well (or) Should I just collect info from internet job portals etc.(i.e., similar to skills.txt, education.txt)? How to proceed with my problem? Any input u can provide would be helpful. $\endgroup$
    – user_12
    Commented May 14, 2020 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.