New answers tagged

0 votes
Accepted

Input length of Sentence BERT

Since BERT is designed for a sentence it captures the context in a sentence. However in a paragraph they will be number of sentences and a number of context. it is not a good idea to insert ...
user avatar
  • 141
2 votes

**tokens when tokens is a dictionary

When **dict is used in a function call like that, the dictionary is expanded into keyword arguments, so the dictionary keys become the keywords and the dictionary values become the argument values. So ...
user avatar
  • 439
2 votes
Accepted

why there is no preprocessing step for training BERT?

Although a definitive answer can only be obtained by actually trying it and it would depend on the specific task where we evaluate the resulting model, I would say that, in general, no, it would not ...
user avatar
  • 16k
0 votes

Why is the decoder not a part of BERT architecture?

I only understood tranformer decoders and encoders after I watched this video. Check it out: https://www.youtube.com/watch?v=0_4KEb08xrE
user avatar
  • 101
2 votes

Usage of Word2Vec

Word2Vec word vectors were being used for classification purposes in the early days. We could take word vectors of all the words in the sentence to get the resultant vector which could give us some ...
user avatar
4 votes

Usage of Word2Vec

Sure, you can average the word2vec vectors of all words in the sentence and train a linear classifier with labeled data. Before doing that, you may remove stopwords that do not add meaning. This ...
user avatar
  • 16k
1 vote
Accepted

Transformers vs RNN basic doubt

There are multiple concepts mixed in your question. Contextual vs. non-contextual word embeddings: word2vec is a non-contextual approach to obtaining token embeddings. This means that a specific word ...
user avatar
  • 16k
1 vote
Accepted

Limitations of NLP BERT model for sentiment analysis

BERT is pre-trained on two generic tasks: masked language modeling and next sentence prediction. Therefore, those tasks are the only things it can do. If you want to use it for any other thing, it ...
user avatar
  • 16k

Top 50 recent answers are included