I am checking this official TensorFlow tutorial on a Transformer model for Portuguese-English translation.
I am quite surprised that when the Transformer is created, their final output is a Dense layer with linear activation, instead of Softmax. Why is that the case? In the original paper Attention is All You Need the image is pretty clear, there is a Softmax layer just at the end (Fig.1, p. 3).
How can you justify this difference, when your task involves building a language model and your Loss is based on sparse categorical crossentropy?