1
$\begingroup$

I am trying to do some NLP on Simplified Chinese texts (needing to extract sentence structure and to do named entity recognition). I've used spaCy previously for English texts, but I see the notes on the Chinese models suggest they are a work in progress, and the NER extraction accuracy has been poor for the examples I've tried.

Which NLP library has the most mature pre-built Chinese language models? Ideally Python based.

$\endgroup$
1
$\begingroup$

There are no common libraries that support high quality named entity recognition for Chinese.

Other options include Information-Extraction-Chinese on GitHub or adapting a paper with code.

| improve this answer | |
$\endgroup$
0
$\begingroup$

After evaluating a few libraries against Chinese texts:

  • spaCy's POS tagging has poor accuracy
  • Stanford Stanza's POS tagging is generally good, but often incorrectly identifies words in single word sentences as POS = "PUNCT"
  • NLTK does not natively handle Chinese POS tagging
  • CoreNLP has good POS tagging accuracy and does a reasonable job of NER tagging.

I suspect this is more about the quality of the pre-built models than the libraries themselves.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.