What is the input to an encoder-decoder transformer in next word prediction task?

Question

I'm trying to understand how encoder-decoder architectures are used, or if they are used at all, for generative tasks that do not require an explicit prompt (ie. machine translation, summarization, etc.).

From my understanding, decoder-only models autoregressively predict the next token in a sequence given its previous predictions. This makes sense, as we can simply keep feeding it tokens already predicted during inference. But how is this done when there is an encoder involved? For machine translation, we have the sequence in the source language to feed to the encoder. Similarly, we can feed it a passage to summarize for summarization. What would we feed the encoder if we simply wanted next word prediction? Do we feed it the sequence we want it to complete? I haven't found any examples of this task being performed. Does this mean that encoder-decoder models aren't needed for this task?

Answer 1

If you just want the model for doing next token prediction, then you would not use an encoder-decoder architecture. Instead, you would use a only the decoder, and feed the text you have to it to get the next word prediction.