6

The maximum input length is a limitation of the model by construction. That number defines the length of the positional embedding table, so you cannot provide a longer input, because it is not possible for the model to index the positional embedding for positions greater than the maximum. This limitation, nevertheless, is not arbitrary, but has a deeper ...


1

For Question Answering, you need 2 logits : one for the start position, one for the end position. Based on these 2 logits, you have an answer span (denoted by the start/end position). In the source code, you have : pooled_output = self.pooler(sequence_output) If you take a look at the pooler, there is a comment : # We "pool" the model by simply taking the ...


Only top voted, non community-wiki answers of a minimum length are eligible