3
$\begingroup$

I need to pull the names of companies out of resumes. Thousands of them. I was thinking of using NLTK to create a list of possible companies, and then cross-referencing the list of strings with something like SEC.gov.

I've already been able to successfully pull the candidate's name, and contact info off of the resumes with some RegEx, but this one has me quite stumped.

What I'm thinking is that I could use NLTK to create a list of strings of proper nouns from the resume's, and then search SEC.gov, or some other database.

This is a link to the SEC page I would be searching: SEC company search page

Read Resume1
Get all potential company names as list of strings potentialCompanies
    IF searching for string1 in SEC gets result, THEN add to candidateCompanies
        ELSE remove from potentialCompanies, go to next string

My Questions

To people that have used NLTK, would there be a better way of getting the potential companies from the text besides using proper nouns?

Would there be a better place to search for companies than the SEC site?

I have never done any web scraping before, and don't really know where to start if it is needed.

(I had posted this on Stack Overflow but they told me that it might be better suited for here...)

$\endgroup$
  • $\begingroup$ Cross posting is generally discouraged in SE. So, please delete one of the two posts. I'm sure you'd get a nice answer here, and welcome to the site! :) $\endgroup$ – Dawny33 Jan 7 '16 at 10:07
  • $\begingroup$ I'll go take the other one down if it's against the rules, interested to hear your guys take on this! :D $\endgroup$ – FishMonkey Jan 7 '16 at 10:13
2
$\begingroup$

NLTK has a built-in NER model that would extract potential Organizations from text, you can read about it here (and see examples) NLTK book (look for section "5 Named Entity Recognition").

However, if your input text has organizations in a very specific context that wasn't seen by NLTK NER model, performance might be quite low. In that case you should be looking into training your own NER model, what would extract company names. For that you would require to manually markup a small amount of your dataset.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.