Transformer models are trained using inputs and outputs. They are both embedded and encoded and used to train multi-head attention mechanisms...
But how can we use a transformer model to predict new data? We won't have any "output" to feed the model yet.
For example you use English and Spanish text to create a dictionary. But when you want to translate new English text you don't know the translation yet.
Note: this answer assumes that the question is about how to use the Transformer model at inference if there is no output to use
At training time, we have the expected output of the model, so there is not a problem, because on the decoder input we use the expected output prefixed with a special <bos> (beginning of sentence) token.
At inference time, we have no output data. Here, we decode one token at a time: first, we pass as decoder input just the <bos> token, and the model generates a single output, which is the prediction for the first token of the output sequence; let's call is $P_1$. Then, we concatenate such predicted token with the sequence used previously as input to the decoder (i.e. [<bos>]), obtaining [$P_1$, <bos>], and we use it as new input to the decoder. In this second decoding step, the decoder generates 2 predictions: [$P_1$, $P_2$]. We take $P_2$ and concatenate it to the previous input, obtaining [<bos>, $P_1$, $P_2$]. We repeat the procedure until we obtain as prediction the <eos> (end of sequence) token, which marks the end of the predicted sequence.
Note: this answer assumes that the question is about a scenario were there is no output data available.
The Transformer model is typically trained using supervised learning, that is you train the model to generate certain output when receiving certain input.
If you don't have input and output data to train, then you can't use supervised learning, not only not for the Transformer, but not for any ML model.
There are other ML paradigms that don't require output data. For instance, reinforcement learning, where you need a "reinforcement signal" that tells the model "how well" it's doing instead of requiring you to provide the expected output. You can review how reinforcement learning has been applied to Transformer models in the survey A Survey on Transformers in Reinforcement Learning. This way of training, however, usually leads to much worse results than supervised learning setups.
This answer is based on Eduardo Munoz's blog "Attention is all you need: Discovering the Transformer paper" in Towards Data Science.
To understand how transformer prediction works, the main thing to remember is that the entire input sequence is used, but the output is the next word or token. And the predict function is called repeatedly until a stopping condition is reached (i.e. the "End of Sequence" token). So, say you've used the model to predict $n$ tokens and now you want to predict the next token ($n+1$). The input to the predict call is the input sequence, plus the sequence of $n$ output tokens that have been generated so far. These $n$ output tokens are used to generate the output embedding and positional encoding that are the input to the decoder.
The transformer model uses a special token "SoS" (start of sequence) to indicate the start of the output sequence, so before the call to predict the first "real" output token, the output sequence is initialised to this "SoS" token.
This works because of the the shifted outputs plus the masked attention layer used in the decoder. When the model is trained, the entire input and output sequences are used. However, the shifted outputs plus the masked attention layer ensure that only output information up to position $i-1$ is used to generate the predicted $i^{th}$ token.
