I am training custom Spacy NER models (v2x). I have come across an interesting problem. There are languages that are morphologically rich. For example, in German: Nominative: der Name, Accusative: den Namen, Genitive: des Namen. (P.s. This is just an example. I am not going to label an entity with its article. :) ) So the question is, if my model only sees "der Name" in examples, how should it be able to recognize "den Namen" or "des Namen"?

Lemmatization is unfortunately out of the question, because it modifies the indexes of start and end characters of entities. Another possibility is to label and train all possible forms of an entity. But then comes another problem: if I label "der Name" and "den Namen", then for Spacy model these are two different entities, right?

How do you deal with this problem? Or any ideas?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.