New answers tagged

1 vote

How does Bert masked language modelling task make sense if half the time the next sentence is wrong context in the sequence passed through the encoder

First, note that the purpose of next sentence prediction objective is not to contribute to the contextual embeddings part, but to allow other downstream tasks like sentence classification and textual ...
noe's user avatar
  • 23.8k

Top 50 recent answers are included