Does ChatGPT use an encoder-decoder architecture, or a decoder-only architecture? I have been coming across Medium and TowardsDataScience articles suggesting that it has an encoder-decoder architecture (see sources below):
I even asked ChatGPT myself whether it uses an encoder and decoder, or just a decoder, and it told me it has both an encoder and decoder. Here is one of its responses to the question:
"As a language model, ChatGPT does not have a decoder-only architecture, it has an encoder-decoder architecture. Having an encoder-decoder architecture enables language models like ChatGPT to process input sequences and generate outputs based on that input. The encoder processes the input sequence and generates a set of hidden states, which are then used by the decoder to generate the output sequence. Having both an encoder and a decoder allows ChatGPT to handle a variety of tasks that involve generating text based on input text, such as language translation, summarization, and answering questions. Having an encoder-decoder architecture allows ChatGPT to maintain context and consistency in the generated output, which is important for many NLP tasks. It is possible to train a decoder-only language model, but it would be limited in its ability to generate coherent text as it would not have access to the context provided by the encoder."
However, I have been under the definite impression for quite some time now that GPT-3 (from which ChatGPT was in part derived) is a decoder-only model. And I take with a grain of salt ChatGPT's explanation of its own architecture given that it seems prone to generating incorrect answers sometimes. Also, with the huge fanfare of ChatGPT and the potential for misinformed authors writing about the model, I wonder if anyone knows of a reliable source that can clarify this question.