1
$\begingroup$

BERT is an NLP model developed by Google. The original BERT model is built by the TensorFlow team, there is also a version of BERT which is built using PyTorch. What is the main difference between these two models?

$\endgroup$

1 Answer 1

7
$\begingroup$

There are not only 2, but many implementations of BERT. Most are basically equivalent.

The implementations that you mentioned are:

These are the differences regarding different aspects:

  • In terms of results, there is no difference in using one or the other, as they both use the same checkpoints (same weights) and their results have been checked to be equal.
  • In terms of reusability, HuggingFace library is probably more reusable, as it is designed specifically for that. Also, it gives you the freedom of choosing TensorFlow or Pytorch as deep learning framework.
  • In terms of performance, they should be the same.
  • In terms of community support (e.g. asking questions in github or stackoverflow about them), HuggingFace library is better suited, as there are a lot of people using it.

Apart from BERT, the transformers library by HuggingFace has implementations for lots of models: OpenAI GPT-2, RoBERTa, ELECTRA, ...

$\endgroup$
3
  • $\begingroup$ What is the difference between these implementation $\endgroup$
    – codeczar
    May 8, 2020 at 2:42
  • $\begingroup$ I updated my answer with explicit differences regarding different aspects. $\endgroup$
    – noe
    May 8, 2020 at 8:34
  • 1
    $\begingroup$ Performance seems to be slightly faster with Pytorch, according to HuggingFace's benchmark : huggingface.co/transformers/benchmarks.html $\endgroup$
    – Astariul
    Jun 23, 2020 at 1:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.