0
$\begingroup$

I read about BERT embedding model and FLAIR embedding model, and I'm not sure I can tell what are the differences between them ?

  • BERT use transformers and FLAIR use BLSTM
  • With BERT, we the feed words into the BERT architecture, and with FLAIR we feed characters into FLAIR architecture.
  1. What are the strengths of BERT embedeeing ?
  2. What are the strengths of FLAIR embedeeing ?
  3. In which cases would we prefer to use one model rather than another ?
$\endgroup$

1 Answer 1

2
$\begingroup$

In general, BERT is probably better for tasks where meaning plays an important role. FLAIR is probably just as good on tasks related to syntax and morphology. Also, the typical advantage of character-level models is their better robustness towards noise (cf. case study in machine translation). There is not much direct comparison, however, there are many indirect clues that suggest so.

FLAIR is evaluated on named entity recognition, chunking, and part-of-speech tagging. All these three tasks rely heavily on syntax. FLAIR reports the F-1 score of 93.09 on the CoNLL-2003 Named Entity Recognition dataset, the same as BERT reports the F1-score of 92.8. (Note, however, that there are BERT-like models that are much better than the original BERT, such as RoBERTa or ALBERT.)

The semantic abilities of pre-trained representations are evaluated in the GLUE and SuperGLUE benchmarks, where BERT is used as a baseline and FLAIR is not in the leaderboards at all. This suggests it would work well. Also, very recent attempts to do character-level pre-trained representations (cf. Charformer, CANINE) struggle to reach good semantic performance, which also suggests that FLAIR as a character-level representation might have problems with semantic-oriented tasks.

To summarize, I would only consider FLAIR for the tasks they test it on (NER, POS tagging, chunking), especially for noisy user-generated data. For everything else I would use RoBERTa or other BERT-like models.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.