I've a problem where I should create a custom NER by using sklearn CRF. In the official tutorial, they are using CoNLL2002 corpus is available in NLTK where the entities are represented with a single word but in my problem, an entity can be formed with multiple words ex: United States of America, Cinema at Miami, etc.
Can CRF handle this?
Absolutely. If you look at the training tutorial, it implies that this isn't an issue at all. When using multi-word entities, you typically need to use a IOB or BILUO tagging schemes, which helps your model train better.
But from a mathematical perspective, there aren't any restrictions for a CRF, as CRF models the likelihood of particular sequences/transitions. Often, people set restrictions for particular transitions if you know in advance that they aren't possible. But by default, all transitions are allowed. In sklearn-crf, allowing all transitions is done by using the all_possible_transitions=True argument.
