4
$\begingroup$

I'm checking the NER FLAIR algorithm with typos:

'Jackson has a number of apartments in Les Angeles San Diego and Oakland'

  • I wrote Les Angele instead of Los Angele
  • I deleted the comma between Les Angele and San Diego

And it seems that FLAIR detectes the correct locations:

Span [1]: "Jackson"   [− Labels: PER (0.9986)]
Span [8,9]: "Les Angeles"   [− Labels: LOC (0.9788)]
Span [10,11]: "San Diego"   [− Labels: LOC (0.9483)]
Span [13]: "Oakland"   [− Labels: LOC (0.9818)]
  1. How FLAIR detects that Les Angeles is place location ?
  2. How it detects that Les Angeles is one location and not 2 (2 words) ?
  3. How it detects that Les Angeles and San Diego are two different locations (there is no comma) ?
$\endgroup$

1 Answer 1

4
$\begingroup$

This is not specific to FLAIR, this is how NER models work in general. A NER model captures the clues in a sentence which are likely to correspond to an entity of a particular category, for example:

  • A location is often preceded by the word in
  • A proper name starts with a capital letter, and most locations are proper names.

These two clues above probably explain (1).

  • After one word has been recognized as part of an entity, it's likely that the next word is also part of the same entity.

This probably explains (2).

I'm not sure how the model gets (3), possibly because it has seen the entity San Diego a few times in the training data.

All these rules have been statistically inferred from the training data during the training stage.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.