I am reviewing various autoencoder setups for MNIST reconstruction, Seq2Seq translation and others. My naive understanding of data flow is as follows:
Input -> [Encoder] -> Hidden Representation -> [Decoder] -> Output.
However, in case of Seq2Seq translation task similar to Sutskever et. al. decoder input is combined from hidden state + input sequence.
I wonder how is the input of decoder dependent on the target task? Why do we need to put input sequence besides the hidden state? Any high-level explanation is appreciated.