2
$\begingroup$

If we treated NER as a classification/prediction problem, how would we handle name entities that weren't in training corpus?

For example, "James was born in England." James was labeled as a PERSON and England as a LOCATION. But we type another completely strange sentence like "Fyonair is from Fuabalada land." We as human can understand Fyonair is a person (or maybe princess from a fairy tale) and Fuabalada is the land where she comes from.

How would our model identify it if it wasn't included in billions of corpus and tokens? Can unsupervised learning achieve this task?

$\endgroup$

2 Answers 2

1
$\begingroup$

If we treated NER as a classification/prediction problem, how would we handle name entities that weren't in training corpus?

The goal of a NER Tagger is to learn patterns in language that can be used to classify words (or more generally, tokens), given a pre-specified set of classes. These classes are defined before training and remain fixed. Classes such as: PERSON, DATETIME, ORGANIZATION, ... you name it.

A good NER Tagger will learn the structure of a language and recognize that "Fyonair is from Fuabalada land." follows some linguistic rules and regularities, and that from these regularities (learned autonomously during training) the classifier can attribute Fyonair class PERSON and to Fuabalada the class LOCATION.


How would our model identify it if it wasn't included in billions of corpus and tokens?

In fact, Deep Learning models tend to work better than others with very large datasets (the so called "big data"). On small datasets they are not extremely useful.


Can unsupervised learning achieve this task?

NER tagging is a supervised task. You need a training set of labeled examples to train a model for that. However, there is some unsupervised work one can do to slightly improve the performance of models. There is this useful paragraph that I took from Geron's book:

Suppose you want to tackle a complex task for which you don't have much labeled training data [...] If you can gather plenty of unlabeled training data, you can try to use it to train an unsupervised model, such as an autoencoder or a generative adversarial network [...] Then you can reuse the lower layers of the autoencoder or the lower layers of the GAN's discriminator, add the output layer for your task on top, and fine tune the final network using supervised learning (i.e. the label training examples).

It is this technique that Geoffrey Hinton and his team used in 2006 and which led to the revival of neural network and the success of Deep Learning.

[ p. 349, 2nd edition. ]

(Best book on Machine Learning ever, IMHO.)

This unsupervised pretraining is the only way to use unsupervised models for NER that I can think of.

Good luck with your task!

$\endgroup$
8
  • $\begingroup$ thank you for the thorough answer pointing out each question segment. does it imply that the language model big enough like BERT can tackle this task in an unsupervised manner if trained on large enough unlabeled corpus? if so, how large the corpus should be trained on for the model to achieve with the decent error rate? just some crumbs of references would be appreciated ... $\endgroup$
    – Kevin
    Jun 6, 2020 at 9:42
  • $\begingroup$ Training BERT is no joke. If you don't have GPUs (or TPUs), or a lot of money to be spent on cloud computing, it would be impossible to do. Usually, BERT is used to generate very fancy word embeddings, to be fed later into the actual model (implemented by you). $\endgroup$
    – Leevo
    Jun 6, 2020 at 9:58
  • $\begingroup$ On dataset size, I would adapt my model size on the size of the datasets available. A good place to start is this Kaggle dataset for NER Tagging. You can check the Notebook produced and their performance, and doing something similar would be a very cool project to start with, IMHO. $\endgroup$
    – Leevo
    Jun 6, 2020 at 9:59
  • $\begingroup$ thanks a million for your helpful thought. i will try it in a way you suggested. the problem with my custom labeled dataset is that the corpus size is too small. the performance of using neural net isn’t significant like u said and the probabilistic approach like CRF isn’t quite reliable for general information retrieval system. the language Burmese is one of the low-resourced languages. that’s why I was thinking of exploiting raw corpus by unsupervised learning to complete this task. $\endgroup$
    – Kevin
    Jun 6, 2020 at 11:02
  • $\begingroup$ on the other hand, quite unfortunately, i don’t huv and cannot afford expensive resources.. so i guess i will just have to implement tiny model depending on my corpus size .. hopefully it is generalized well enough for my desired use case .. $\endgroup$
    – Kevin
    Jun 6, 2020 at 11:04
1
$\begingroup$

First, when we as humans interpret "Fyonair is from Fuabalada land" we use our knowledge of "is from" and "X land" to infer that Fyonair is probably a person and Fuabalada land probably a location. Therefore our process is not (at least not completely) unsupervised: we have seen this sentence structure before ("training") and we use our "model" to "predict" the two entities.

A good (supervised) NER is trained to recognize this kind of patterns as well. The example that you mention might be a bit too hard, but with something like "Dr Fyonair is the new CEO of Fubalada" a decent NER should be able to recognize that "Dr Fyonair" is a person (thanks to the "Dr") and that "Fubalada" is a company (thanks to the "CEO of") even though it has never seen these particular proper names.

So a standard (supervised) NER is supposed to recognize entities it has never seen before, provided it has clues in the sentence about them. Technically it's not really a state of the art NER if it recognizes only the entities that it has seen during training, it's just a string matching program. It's true that NERs make a lot of errors with entities which haven't been seen previously, but that's simply because they are harder to catch in general.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.