Tokenizer for GPT-3 is the same as GPT-2: https://huggingface.co/docs/transformers/model_doc/gpt2#gpt2tokenizerfast linked via: https://beta.openai.com/tokenizer --- **UPDATE** March 2023 For newer models, including GPT-3.5 (turbo), GPT-4, and latest embeddings, use `tiktoken` tokenizer with the `cl100k_base` encoding: https://github.com/openai/tiktoken A full model-to-encoding mapping can be found [here](https://github.com/openai/tiktoken/blob/main/tiktoken/model.py)