6
$\begingroup$

I'm trying to use MITIE to extract named entities from short text. I'm interested in entities such as dates, times, names, and locations. Out of the box, MITIE only recognises names, locations, and organisations. I'd like to train it to recognise dates, times and other categories as well. From looking at the structure of MITIE's directories and from the dlib website, I gather that this is done via an SVM. Is this correct?

With regards to adding new categories to the named entity recogniser, I have several questions:

  1. Can this be done in an augmentative fashion? That is, given an existing NER system, can I just add categories examples and train it to recognise those as well? Or do I need to train models from scratch?
  2. If I do need train models from scratch, what dataset can I use to do this?
  3. Related to adding new examples, is there an online method that I can use, feeding the system new examples and categories as and when I need to?
Improve this question
$\endgroup$
2
  • $\begingroup$ blog.booking.com/named-entity-classification.html This might not answer all of your questions, but definitely worth reading the article mentioned above. $\endgroup$
    – rbhat
    Jul 11, 2017 at 8:58
  • $\begingroup$ Thanks @user2997081 - that's where I started my search, and although the article is good, it took me a fair amount of searching to understand MITIE better. I hope my answer can be of help to others searching along similar lines. $\endgroup$
    – Navaneethan Santhanam
    Jul 14, 2017 at 6:46

1 Answer 1

Reset to default
6
$\begingroup$

After having used MITIE for a few weeks, I feel like I at least have enough to answer my basic questions:

  1. (and 3.) All models need to be trained from scratch - there is no online method to add new samples to the model as they come in. This is unfortunate because MITIE takes at least 45 minutes to an hour to train on a ~20k-sized dataset.
  2. The datasets I used were ATIS, CoNLL 2003, and DBpedia

I've found MITIE to be quite good as far as classification accuracy goes, although it takes a bit of work to prepare datasets for it.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Could you please add a small reproducible example on how to use MITIE for NER & POS in Python. Their github is rather unclear and confusing. $\endgroup$
    – kRazzy R
    Oct 4, 2017 at 22:50
  • $\begingroup$ @kRazzyR it'll take me at least a few days to put this together. In the meantime, have a look at this example (for NER) - github.com/mit-nlp/MITIE/blob/master/examples/python/…. I found it to be quite straightforward, and it's only a few lines once you've cloned their github repo and downloaded the necessary files (from their README). Let me know if that works for you. If not, I'll put together an example. I've not used MITIE for POS (not sure they even have support for POS), actually. $\endgroup$
    – Navaneethan Santhanam
    Oct 5, 2017 at 10:51
  • $\begingroup$ If you want, Spacy is great for dates, names, and somewhat for times. MITIE is better-ish for locations. Addresses are always awkward. Combining the two may work without needing a ton of training data. $\endgroup$
    – Andrew Scott Evans
    Aug 23, 2019 at 5:08
  • $\begingroup$ "quite good for classification accuracy" doesn't say much: how many classes? what accuracy? and how did that compare to other specific classifiers on the same task? $\endgroup$
    – smci
    Apr 1, 2021 at 18:31
  • $\begingroup$ Sorry, it's been nearly 4 years since I did this. I'm no longer at that company, so I don't have access to data. The classes I used for training were - Location, Person, Organisation. I compared it to spaCy, Stanford NLP, and Vowpal Wabbit, and it performed better than all of them. I think spaCy was a not-too-close second. IIRC, precision was in the 80-odd percentage range for the test set, dropping to the 70% range in production. $\endgroup$
    – Navaneethan Santhanam
    Apr 3, 2021 at 8:57

Not the answer you're looking for? Browse other questions tagged or ask your own question.