Understanding the generality of the NER problem

Question

Named-entity recognition (NER) is a well-known problem in the NLP literature.

It typically addresses the problem to locate and classify named entities in text, e.g. Organizations and Products.

I'm trying to solve a similar problem but, in my view, a bit more general. Given an input text, I want to be able to comprehensively annotate the whole text; not only specific entities like Actors and Organizations but also higher-level concepts like Conditions of Applicability and Temporal Conditions, e.g.:

The added difficulty is that we have nested "entities", e.g. (from above):

<denotic> must, <temporal> within the specified period </temporal>, notify ... </deontic>

Can this still be formulated as a NER problem? If so, what would be the best type of model to solve this task assuming a dataset of ~ 50 K examples?

Answer 1

The problem described is not a more general version of Named Entity Recognition, it is a different problem called parsing. Parsing consists in extracting the syntactic structure of a text, usually in the goal to better capture its semantics. There are various approaches:

• Shallow parsing only identifies the constituents of the sentences (based on your example this could be sufficient in your case)
• Statistical parsing and in particular Dependency parsing represent the full structure of the sentence, including the links between its constituents.

There are various libraries and datasets for parsing: one of the most famous is probably the Stanford parser, but there are many others often included in NLP toolkits such as OpenNLP. The Universal Dependencies project is a vast multilingual collection of annotated text which can be used to train parsers.

Semantic Role Labeling (SRL) is a closely related task which consists in identifying the semantic relations between a predicate (verb) and its related constituents (e.g. subject, object).