Let's say I have a resume and I have segmented the work section.

Usually work section of resume contains company name, designation, work period and job description. Now I have 1000's of resumes and I have manually annotated the work section of each resume with those 4 labels.

But the problem here is each work section is quite large around 1400- 3000 words? Also job-description annotation is not one word like company name etc... the entire responsibilities of the job is annotated as one entity.

So, will this work? please let me know what are the things I should consider for such NER entity extraction?

Any suggestions would really help. Thank you in advance.


1 Answer 1

  • The size of the document on which NER is run shouldn't be a problem at all, a standard NER system scans the document sequentially and just marks any entity it finds.
  • The size of the entities to find might be more of an issue, because typical NER systems rely on the previous few words to detect the boundaries of an entity. If the entity spans a large sequence of text, it's harder for the system to detect where it ends. In case it actually causes a problem with your data, it might be possible to specify exactly which features to use in the CRF model (this depends on the implementation I guess).
  • $\begingroup$ Are there any solutions I could use regarding the size of entities? because in my case all the data in the work section comes under each entity. There aren't any -Outside entity. The job description is what feeling tricky because it's very large so finding boundaries is making it hard. $\endgroup$
    – user_12
    Jul 2, 2020 at 16:02
  • 1
    $\begingroup$ @user_12 it might work well even in this case, there's no way to know before testing it: assuming the model is trained on enough examples so it will have seen many of these long entities, the probability of "inside followed by inside" will be quite high. So the model will not shorten the entity just because it's long, however it might not be successful at catching the end unless there's a clear trigger. In a fully configurable CRF model you can add any number of specific features, so you could try to help the model this way. But as I said test it first, it might work well immediately. $\endgroup$
    – Erwan
    Jul 2, 2020 at 16:26
  • $\begingroup$ Thank you so much for responding. I'll try it out and see. $\endgroup$
    – user_12
    Jul 2, 2020 at 17:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.