In BERT, NSP (Next Sentence Prediction) is for predicting next sentence based on context and Text prediction task is also for predicting next word or phrases.
So, both are for predicting next sentence or word/ phrase only and both are BERT NLP tasks, then why these two?
NSP is not for predicting the next sentence but a binary classification task to predict whether the second sentence was originally immediately after the first or not:
During BERT's training, it is given two concatenated text sentences as input. Some tokens (15%) of the sentences are masked (the original tokens are replaced by a special [MASK] token). BERT's training objective has two parts:
To guess the original tokens at the masked positions. This objective is called "masked language modelling".
To determine if the second segment was following the first segment in the original document it was extracted from (i.e. true/false). This training objective is the "Next Sentence Prediction (NSP)" task.
The outputs of the masked language model task are at the positions of the masked tokens, while the output at the first position (i.e. the artificial [CLS] token added at the beginning of the text) is used for the NSP task:
