Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I read about BERT embedding model and FLAIR embedding model, and I'm not sure I can tell what are the differences between them ?
BERT use transformers and FLAIR use BLSTMBERT, we the feed words into the BERT architecture, and with FLAIR we feed characters into FLAIR architecture.BERT embedeeing ?FLAIR embedeeing ?In general, BERT is probably better for tasks where meaning plays an important role. FLAIR is probably just as good on tasks related to syntax and morphology. Also, the typical advantage of character-level models is their better robustness towards noise (cf. case study in machine translation). There is not much direct comparison, however, there are many indirect clues that suggest so.
FLAIR is evaluated on named entity recognition, chunking, and part-of-speech tagging. All these three tasks rely heavily on syntax. FLAIR reports the F-1 score of 93.09 on the CoNLL-2003 Named Entity Recognition dataset, the same as BERT reports the F1-score of 92.8. (Note, however, that there are BERT-like models that are much better than the original BERT, such as RoBERTa or ALBERT.)
The semantic abilities of pre-trained representations are evaluated in the GLUE and SuperGLUE benchmarks, where BERT is used as a baseline and FLAIR is not in the leaderboards at all. This suggests it would work well. Also, very recent attempts to do character-level pre-trained representations (cf. Charformer, CANINE) struggle to reach good semantic performance, which also suggests that FLAIR as a character-level representation might have problems with semantic-oriented tasks.
To summarize, I would only consider FLAIR for the tasks they test it on (NER, POS tagging, chunking), especially for noisy user-generated data. For everything else I would use RoBERTa or other BERT-like models.
