In the "Attention Is All You Need" paper, the decoder consists of two attention sub-layers in each layer followed by a FF sub-layer. The first is a masked self attention which gets as an input the output of the decoder in the previous step (and the first input is a special start token). The second, 'encoder-decoder', attention sub-layer gets as an input queries from the lower self-attention sub-layer and keys & values from the encoder. I do not see the use of the output of the FF sub-layer in the encoder; can someone explain where is it used? Thanks
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
We can see this in the original Transformer diagram:
The output of the last encoder FF layer is added to the original input of the same layer, then layer normalization is applied and that is the output of the whole encoder, which is used as keys and values for the encoder-decoder attention blocks in the decoder.
-
$\begingroup$ "which is used"; how exactly is it used? i.e., the last encoder, as all the encoders below it, outputs (using Q,K,V matrices) a sequence 'Z'. Now, the decoder uses two types of attention mechanisms in each layer: the first is self attention which DOES NOT take an input directly from the encoder at all. The second is encoder-decoder attention which takes as an input K,V directly from the encoder. In both cases, 'Z' is not used by the decoder... So where is this 'Z' used? Only in upper layers inside the encoder itself? Thanks $\endgroup$ Commented May 16, 2021 at 7:41
-
$\begingroup$ The keys and values of the encoder-decoder attention blocks in the decoder are the output of the last layer of the encoder, as shown in the figure. $\endgroup$– noeCommented May 16, 2021 at 7:55
-
$\begingroup$ I don't know what you are referring to as 'Z'. I think the figure is very informative and complete. Which of the elements of the figure are you assigning that 'Z' ? $\endgroup$– noeCommented May 16, 2021 at 10:43
-
$\begingroup$ OK, I think I got it. I did not ask about the keys and values (which are indeed the input to the 'encoder-decoder' attention sub-layer of the decoder), but about the sequence (I denoted by 'Z' above) which is generated in the second sub-layer of each encoder, i.e. after FF, Add & Norm; I wondered where it is used. So probably this sequence is used only in the encoders, that is, each encoder outputs a sequence which is used as an input to the subsequent encoder layer; except for the last one, which as you wrote @noe, outputs the keys and values. Thanks. $\endgroup$ Commented May 16, 2021 at 14:49
-
$\begingroup$ I'm glad that the answer could help. Please, consider marking the answer as correct with the tick (✓ ) button. $\endgroup$– noeCommented May 16, 2021 at 15:53