# What are the inputs to the first decoder layer in a Transformer model during the training phase?

I am trying to wrap my head around how the Transformer architecture works. I think I have a decent top-level understanding of the encoder part, sort of how the Key, Query, and Value tensors work in the MultiHead attention layers. What I am struggling with is the decoder part, specifically the inputs to the very first decoder layer.

I understand that there are two things. The output of the final encoder layer, but before that an embedded (positional encoding + embedding) version of... well something.

In the original paper in Figure 1, they mention that the first decoder layer input is the Outputs (shifted right). I am a little confused on what they mean by "shifted right", but if I had to guess I would say the following is happening

Input: <Start> How are you <EOS> Output: <Start> I am fine <EOS>

and so the input to the first decoder layer will be [<Start> I am fine].

What is the need for shifting the sequence? Why would we not just input the target sequence itself? I was thinking maybe it's because of the auto-regressive nature of the decoder part, but then the only difference between the sequences would be the <EOS> token if I am seeing this correctly.

As you can probably tell I am a little bit confused by how some of the parts work, so any help to get to a better understanding would be much appreciated.

• The source sequence would be How are you <EOS>
• The input to the encoder would be How are you <EOS>. Note that there is no <start> token here.
• The target sequence would be I am fine <EOS> . The output of the decoder will be compared against this in the training.
• The input to the decoder would be <start> I am fine <EOS>.
• Yes, you are right, the sequence should actually be <start> I am fine. Note that in some implementations, instead of using a specific token for <start>, the <eos> token is reused, and in those cases the sequence would be <eos> I am fine. This is the case of fairseq; you can check their implementation, which is quite clear. – noe Feb 5 at 20:59