i have i a big trouble. I don't understand transformers. I understand embedding, rnn's, GAN's, even Attention. But i don't understand transformers. Approximately 2 months ago i decided to avoid usage of transformers, because i found them hard. But i can't anymore avoid transformers. Please, help me. I want to use and understand work of transformers. How can i start to work with them?Past the fact that i want to understand their idea in general, i also want to can write/implement them using keras/tensorflow Of course i tied to read some tutorials. But i don't understand them anyway.
These are the answers to the specific doubts that you pointed out in the comments:
Transformers use many building blocks, like self-attention, layer normalization, residual connections, etc. Tutorials like The illustrated transformer are very useful to understand these blocks and how they fit together.
The role of the softmax is to normalize the sum up to 1. In the example, you can see that the softmax-normalized values are 0.88 and 0.12, which add up to 1. The result of the softmax is then used as weights for the values, which are then added together.
The decoder is very similar to the encoder, especially at training time. The main differences are that the queries are taken from the target side while keys and values are from the source side and that the attention is masked to avoid the prediction for time t to be dependent on the tokens at the same and future positions.
The decoder receives both the output of the encoder and the target sequence, either the full sequence at training time or the partial sequence at inference time.
At training time, the decoder receives the whole target sentence tokens. At inference time, we don't have the target sentence; instead, we use the model autoregressively: at each decoding step we pass as input the previous predictions, get the prediction for the next token, concatenate it with the previous step input and use it as input for the next step; at the first step of the autoregressive decoding we simply pass as input a sequence with just the special token