The CoreNLP parts of speech tagger and name entity recognition tagger are pretty good out of the box, but I'd like to improve the accuracy further so that the overall program runs better. To explain more about accuracy -- there are situations in which the POS/NER is wrongly tagged. For instance:

  • "Oversaw car manufacturing" gets tagged as NNP-NN-NN

Rather than VB* or something similar, since it's a verb-like phrase (I'm not a linguist, so take this with a grain of salt).

So what's the best way to accomplish accuracy improvement?

  • Are there better models out there for POS/NER that can be incorporated into CoreNLP?
  • Should I switch to other NLP tools?
  • Or create training models with exception rules?

1 Answer 1


Your best best is to train your own models on the kind of data you're going to be working with.

  • $\begingroup$ I think this would be more helpful to recommend particular models or tools in answer to this question. $\endgroup$
    – Sean Owen
    Commented Sep 16, 2014 at 15:54
  • $\begingroup$ I haven't seen any publicly-available models for Stanford NER, other than those distributed by the Stanford NLP Group itself. These include multiple versions for English (including case-insensitive models), as well as models for German, Spanish, and Chinese. nlp.stanford.edu/software/CRF-NER.shtml#Models However, these models were trained mostly on annotated news articles, which may not work well on other kinds of text data. Here's how you would train your own models for Stanford NER: nlp.stanford.edu/software/crf-faq.shtml#a $\endgroup$ Commented Sep 18, 2014 at 13:31
  • $\begingroup$ Also, other NER tools offer languages for other models. For example, Apache OpenNLP offers a model trained on Dutch: opennlp.sourceforge.net/models-1.5 But again, in the long run, the way to get the best performance on your own data is to train custom models on your own data. It takes a TON of work to acquire the training data and painstakingly annotate it by hand, but it might be worth it in the end. Once you have the annotated training data, most NER tools have methods for training new custom models on that data. $\endgroup$ Commented Sep 18, 2014 at 13:39
  • $\begingroup$ Thanks @CharlieGreenbacker - what if I'm looking for models to use in the corporate environment? Meaning, if people talk about their work, teams, etc. Are there existing work around this domain? $\endgroup$ Commented Sep 23, 2014 at 17:47
  • $\begingroup$ @UzumakiNaruto The same goes here for really any domain, regardless of topic, etc. -- collect a LOT of relevant real-world data and annotate it manually... this will be the training data you use to train new models. Also, you might want to look into the work of Giuseppe Carenini at the Univ of British Columbia -- he's done some research applying NLP, etc. to things like business meeting notes: cs.ubc.ca/~carenini $\endgroup$ Commented Sep 29, 2014 at 17:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.