27
$\begingroup$

I'm looking to use google's word2vec implementation to build a named entity recognition system. I've heard that recursive neural nets with back propagation through structure are well suited for named entity recognition tasks, but I've been unable to find a decent implementation or a decent tutorial for that type of model. Because I'm working with an atypical corpus, standard NER tools in NLTK and similar have performed very poorly, and it looks like I'll have to train my own system.

In short, what resources are available for this kind of problem? Is there a standard recursive neural net implementation available?

Improve this question
$\endgroup$
3
  • $\begingroup$ Have you tried training the Stanford NER on your corpus? There's a tutorial here. $\endgroup$
    – Emre
    Commented Jun 19, 2014 at 20:58
  • $\begingroup$ I have not -- should give that a go to see how it fares. $\endgroup$
    – Madison May
    Commented Jun 19, 2014 at 21:30
  • $\begingroup$ I'd like to use word2vec features or similar, though, since I have access to a relatively small labeled dataset and need to make the most of unlabeled data I have on hand. $\endgroup$
    – Madison May
    Commented Jun 19, 2014 at 21:37

4 Answers 4

Reset to default
6
$\begingroup$

Instead of "recursive neural nets with back propagation" you might consider the approach used by Frantzi, et. al. at National Centre for Text Mining (NaCTeM) at University of Manchester for Termine (see: this and this) Instead of deep neural nets, they "combine linguistic and statistical information".

Improve this answer
$\endgroup$
7
$\begingroup$

Two recent papers use a Deep learning architecture called CharWNN to address this problem. CharWNN was first used to get state of the art results (without handcrafted features) on Part of Speech (POS) tagging on an English corpus.

The second paper by the same author uses the same (or similar) architecture for predicting whether a word belongs to 10 Named Entity classes, with apparent state of the art results.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Interesting. Thanks for sharing this. I wonder why they didn't test this approach on english corpora. $\endgroup$
    – MaticDiba
    Commented Sep 14, 2015 at 11:33
3
$\begingroup$

Try this. This has an implementation of Word2Vec used instead of Bag of Words for NER and other NLP tasks.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ I've had no problem finding word2vec implementations, but I have been unable to find a working recursive net to use. $\endgroup$
    – Madison May
    Commented Aug 28, 2014 at 16:57
  • $\begingroup$ Link is no more active, if possible kindly share the new working link $\endgroup$
    – Amandeep
    Commented Mar 15, 2019 at 10:51
2
$\begingroup$

Here are some ideas about how to use word vectors for NER, which adopts a mostly unsupervised word2vec-centric approach.

  1. Given a set of word vectors (either something you trained, or something off-the-shelf like GoogleNews-vectors-negative300.bin), discover clusters in the vector space. These clusters are basically your definitions for various unnamed concepts.
  2. With minimal supervision, you can map/convert the unnamed clusters to match human knowledge, thus creating named concepts grounded on top of the known word vectors and unnamed concepts. For example, a method findCluster(['joy', 'surprise', 'disgust', 'trust', 'fear', 'sadness', 'anger', 'anticipation']) might return a list containing hundreds of words mostly related to emotion. If you name this list 'emotion', then there you have a named concept 'emotion' defined based on the vector space.
  3. You can also do vector math to find the intermediate concept between two given ones. For example, the vector math might tell you that when given two words 'surprise' and 'disgust', the following are found between the two: dismay, astonishment, disappointment, amazement, bewilderment, exasperation, disbelief, shock, etc. This allows you to build up relationship between concepts.
  4. You can repeat the above to build up various types of named concpts, like: weekdays, all emotions, happy emotions, vehicles, etc.
  5. Once you have built up layers of named concepts, you can then train a RNN on a text corpus that has been augmented with the named concepts, so 'brown fox jumps' is also '{color} {animal} {action}' etc. This way the RNN should be able to learn some rudimentary grammar in an unsupervised manner.
  6. If you have built up a sufficiently powerful grammar from the above, then you should be able to apply it toward some of your NER tasks.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.