I'm trying to extract percentages using Stanford NER. But it is not extracting percentage properly.

import os
from nltk.tag import StanfordNERTagger
os.environ['CLASSPATH'] = 'stanford-ner-2015-12-09/stanford-ner.jar'
os.environ['STANFORD_MODELS'] = 'stanford-ner-2015-12-09/classifiers'

inp_str = 'total revenue received was one hundred and twenty five percent 125% for last financial year'
split_inp_str = inp_str.split()
st = StanfordNERTagger('english.muc.7class.distsim.crf.ser.gz')

This gives following output

[('total', 'O'), ('revenue', 'O'), ('received', 'O'), ('was', 'O'), ('one', 'O'), ('hundred', 'O'), ('and', 'O'), ('twenty', 'O'), ('five', 'PERCENT'), ('percent', 'PERCENT'), ('125%', 'O'), ('for', 'O'), ('last', 'O'), ('financial', 'O'), ('year', 'O')]

Expected ouput:

[('total', 'O'), ('revenue', 'O'), ('received', 'O'), ('was', 'O'), ('one', 'PERCENT'), ('hundred', 'PERCENT'), ('and', 'PERCENT'), ('twenty', 'PERCENT'), ('five', 'PERCENT'), ('percent', 'PERCENT'),('125%', 'PERCENT'), ('for', 'O'), ('last', 'O'), ('financial', 'O'), ('year', 'O')]

Why is it not extracting 125% or one hundred and twenty five percent?

  • $\begingroup$ welcome to data science SE. Could you copy paste a full working example and your expected output? $\endgroup$
    – Stereo
    Commented Jan 9, 2017 at 16:54
  • $\begingroup$ @Stereo Thanks . I have added working code as well as expected output. $\endgroup$ Commented Jan 10, 2017 at 6:34
  • $\begingroup$ It is not predicting the percentage entity every time? $\endgroup$ Commented Jan 10, 2017 at 7:05
  • $\begingroup$ @HimanshuRai The output is coming as five percent instead of expected output $\endgroup$ Commented Jan 10, 2017 at 7:11
  • $\begingroup$ No, I mean did you try for other examples or this is the only sample you tried it for? $\endgroup$ Commented Jan 10, 2017 at 7:12

1 Answer 1


Looking at your code, I dont believe there is any mistake in it. The Stanford NER tagger is modelled by a Conditional Random Field Classifier and is hence bound to make mistakes. If you are able to check for a large number of samples and calculate an accuracy metric on the basis of that you might get a good idea if the pre-trained model is useful for you. Else, I suggest that you train your own model using. The stanford NLP group provide an implementation for training, you just have to give it the data in the required format. Look at this blog to get a good idea on how to do it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.