The currently dominant NMT paradigm is based on an encoder-decoder architecture where the translation is generated autoregressively. This means that each token in the translation is generated conditioning not only on the source sentence but also on the previously generated tokens. The encoder generates a representation of the source sentence in one go and then the decoder takes this representation and generates the target side tokens one by one. This way, the decoder is responsible for the autoregressive part.
When you have an architecture where there is no decoder, you have non-autoregressive (NAR) NMT. It is possible to build a NAR model, but most attempts to train models directly in such a way have failed. They usually repeat the same words once and again, utterly generating garbage instead of actual translations.
Nevertheless, if NAR NMT were possible, it would lead to huge speedups ($O(1)$ instead of $O(n)$ complexity). This is currently a very active line of research. Most of the proposed attempts rely on creating some sort of intermediate (latent) representation to feed to the decoder, which then decodes non-autoregressively.
These are some of the latest articles from that area:
