1
$\begingroup$

I'm training a custom Spacy model. Does it affect the results whether I leave entities unlabeled? For example, Germany is LOC. In one example (1) I label this. In the other example (2) I ignore it and label another entity:

Example 1: Germany (LOC) is a country in Central and Western Europe.

Example 2: Germany borders Denmark (LOC) to the north.

It's not about one entity that is occasionally ignored, but about several.

$\endgroup$
1
$\begingroup$

Yes, it matters. A lot. You need to label every entity you encounter in each sentence. As long as they are non-overlapping, you can add as many entity types and entities per document as you'd like.

First of all, it matters because, in your example, your model receives one example of "Germany" that is a LOC and one that isn't. So, it must mean that the context around "Germany" determines whether it is an entity. Not labeling your entities in some sentences works against the very task you are trying to learn.

In their documentation, they show how to define training data with multiple entities of the same type per document:

TRAIN_DATA = [
    ("Who is Shaka Khan?", {"entities": [(7, 17, "PERSON")]}),
    ("I like London and Berlin.", {"entities": [(7, 13, "LOC"), (18, 24, "LOC")]}),
]
$\endgroup$
1
  • 1
    $\begingroup$ Thank you, @ValentinCalomme! It helps a lot. $\endgroup$
    – gzkhv
    Jan 18 '21 at 8:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.