Questions tagged [bert]
BERT stands for Bidirectional Encoder Representations from Transformers and is designed to pre-train deep bidirectional representations by jointly conditioning on both left and right context in all layers
357
questions
1
vote
1
answer
513
views
BERT has a non deterministic behaviour
I am using the BERT implementation in https://github.com/google-research/bert for feature extracting and I have noticed a weird behaviour which I was not expecting: if I execute the program twice on ...
7
votes
1
answer
9k
views
How is WordPiece tokenization helpful to effectively deal with rare words problem in NLP?
I have seen that NLP models such as BERT utilize WordPiece for tokenization. In WordPiece, we split the tokens like playing to play and ##ing. It is mentioned that it covers a wider spectrum of Out-Of-...
- 696
2
votes
0
answers
215
views
Paragraph Generator using BERT or GPT
I am trying to generate similar sentences, called paragraph generation. For example, what is the name of the eldest brother of ram? - For these paragraphs can be - who is the oldest brother of ram? , ...
- 894
2
votes
1
answer
2k
views
Incrementally Train BERT with minimum QnA records - to get improved results
We are using Google BERT for Question and Answering. We have fine tuned BERT with SQUAD QnA release train data set (https://github.com/google-research/bert , https://rajpurkar.github.io/SQuAD-explorer/...
- 894
3
votes
1
answer
2k
views
How Transformer is Bidirectional - Machine Learning
Asking question in datascience forum, as this forum seems well suited for data science related questions: https://stackoverflow.com/questions/55158554/how-transformer-is-bidirectional-machine-learning/...
- 894
9
votes
1
answer
5k
views
Bert Fine Tuning with additional features
I want to use Bert for an nlp task. But I also have additional features that I would like to include.
From what I have seen, with fine tuning, one only changes the labels and retrains the ...
- 193
21
votes
1
answer
11k
views
Can BERT do the next-word-predict task?
As BERT is bidirectional (uses bi-directional transformer), is it possible to use it for the next-word-predict task? If yes, what needs to be tweaked?
- 691