1
$\begingroup$

Named-entity recognition (NER) is a well-known problem in the NLP literature.

It typically addresses the problem to locate and classify named entities in text, e.g. Organizations and Products.

enter image description here

I'm trying to solve a similar problem but, in my view, a bit more general. Given an input text, I want to be able to comprehensively annotate the whole text; not only specific entities like Actors and Organizations but also higher-level concepts like Conditions of Applicability and Temporal Conditions, e.g.:

enter image description here

The added difficulty is that we have nested "entities", e.g. (from above):

<denotic> must, <temporal> within the specified period </temporal>, notify ... </deontic>

Can this still be formulated as a NER problem? If so, what would be the best type of model to solve this task assuming a dataset of ~ 50 K examples?

$\endgroup$
1
$\begingroup$

The problem described is not a more general version of Named Entity Recognition, it is a different problem called parsing. Parsing consists in extracting the syntactic structure of a text, usually in the goal to better capture its semantics. There are various approaches:

  • Shallow parsing only identifies the constituents of the sentences (based on your example this could be sufficient in your case)
  • Statistical parsing and in particular Dependency parsing represent the full structure of the sentence, including the links between its constituents.

There are various libraries and datasets for parsing: one of the most famous is probably the Stanford parser, but there are many others often included in NLP toolkits such as OpenNLP. The Universal Dependencies project is a vast multilingual collection of annotated text which can be used to train parsers.

Semantic Role Labeling (SRL) is a closely related task which consists in identifying the semantic relations between a predicate (verb) and its related constituents (e.g. subject, object).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.