Questions tagged [attention-mechanism]
The attention-mechanism tag has no usage guidance.
2
questions
35
votes
3answers
25k views
What is the positional encoding in the transformer model?
I'm trying to read and understand the paper Attention is all you need and in it, there is a picture:
I don't know what positional encoding is. by listening to some youtube videos I've found out that ...
14
votes
1answer
5k views
Can BERT do the next-word-predict task?
As BERT is bidirectional (uses bi-directional transformer), is it possible to use it for the next-word-predict task? If yes, what needs to be tweaked?
