Questions tagged [openai-gpt]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
1 answer
4k views

How does BERT and GPT-2 encoding deal with token such as <|startoftext|>, <s>

As I understand, GPT-2 and BERT are using Byte-Pair Encoding which is a subword encoding. Since lots of start/end token is used such as <|startoftext|> and , as I image the encoder should encode ...