3 votes
Accepted

What do these terms mean in the context of Roberta?

Fine-tuning means that you modify the model to adapt it to your data. Feature extraction means that you don't modify the model. In the case of RoBERTa, it means that you feed your data to the model, ...
noe's user avatar
  • 25.6k
1 vote
Accepted

What are the differences between Embedding Layer and Roberta Embedding?

An embedding layer is just a building block to be used as part of neural architectures. It is just a lookup table whose purpose is to represent tokens as vectors, and to learn these vectors as part of ...
noe's user avatar
  • 25.6k
1 vote
Accepted

What are the differences between contextual embeddings of Bidirectional-LSTM and Transformer?

Here are some differences: Computational complexity: LSTMs have linear complexity $O(n)$, because you need to process input tokens one by one, while transformers have constant $O(1)$ complexity ...
noe's user avatar
  • 25.6k
1 vote

Is vision transformer (ViT) always better than CNN?

There are CNN models which can match Transformers. See ConvNext, EfficientNet, etc. ViT is also an old vision transformer architecture. Swin transformers are more performant nowadays.
Cory Fan's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible