New answers tagged

3

Some points first: BERT is a word embedding: BERT is both word and sentence embedding. It needs to be taken into account that BERT is taking the sequence of words in a sentence into account which gives you a richer embedding of words in a context but in classic embeddings (yes, after BERT we can call others "classic"!) you mostly deal with neighborhood i.e. ...


0

When you run BERT, you get one vector per input token + 1 special token called [CLS] + 1 special token called [SEP]. Maybe more precise than calling BERT embeddings as embeddings, would be calling them hidden states of BERT. The contextual information get into the embeddings via 12 layers of self-attentive neural network. However, the tokenization is tricky ...


Top 50 recent answers are included