I know GPT is a Transformer-based Neural Network, composed of several blocks. These blocks are based on the original Transformer's Decoder blocks, but are they exactly the same?
In the original Transformer model, Decoder blocks have two attention mechanisms: the first is pure Multi Head Self-Attention, the second is Self-Attention with respect to Encoder's output. In GPT there is no Encoder, therefore I assume its blocks only have one attention mechanism. That's the main difference I found.
At the same time, since GPT is used to generate language, its blocks must be masked, so that Self-Attention can only attend previous tokens. (Just like in Transformer Decoders.)
Is that it? Is there anything else to add to the difference between GPT (1,2,3,...) and the original Transformer?