6
The maximum input length is a limitation of the model by construction. That number defines the length of the positional embedding table, so you cannot provide a longer input, because it is not possible for the model to index the positional embedding for positions greater than the maximum.
This limitation, nevertheless, is not arbitrary, but has a deeper ...
1
A recent PR changed the location of the scripts you are looking for to examples/legacy/question-answering
1
For Question Answering, you need 2 logits : one for the start position, one for the end position. Based on these 2 logits, you have an answer span (denoted by the start/end position).
In the source code, you have : pooled_output = self.pooler(sequence_output)
If you take a look at the pooler, there is a comment :
# We "pool" the model by simply taking the ...
Only top voted, non community-wiki answers of a minimum length are eligible
Related Tags
huggingface × 17bert × 9
nlp × 8
transformer × 7
question-answering × 5
tensorflow × 4
pytorch × 3
python × 2
tokenization × 2
training × 1
overfitting × 1
transfer-learning × 1
language-model × 1
stanford-nlp × 1
automatic-summarization × 1
weight-initialization × 1
historgram × 1
text-classification × 1
pruning × 1
bart × 1