I am referring to seq2seq model's example code in keras (https://github.com/fchollet/keras/blob/master/examples/addition_rnn.py). The model is :
model = Sequential() model.add(RNN(HIDDEN_SIZE, input_shape=(MAXLEN, len(chars)))) model.add(RepeatVector(DIGITS + 1)) for _ in range(LAYERS): model.add(RNN(HIDDEN_SIZE, return_sequences=True)) model.add(TimeDistributed(Dense(len(chars)))) model.add(Activation('softmax'))
In this model we are passing the encoded input vector from encoder's last state to each time step in the decoder.
Now we are not passing any other input to the decoder except the encoded input vector, but in all seq2seq models we pass output sequence also (time delayed) with the encoded input.
How is this a valid seq2seq model? To my surprise it works well. How does this works?