0
$\begingroup$

So in seq-to-seq models for say NMT, the decoder is a sequence model for the right-shifted intended output. My question is, during training, are the inputs and outputs of the decoder supposed to be the desired labels? Or are just the outputs the desired labels and the inputs are the actual predicted output from the last timestep?

$\endgroup$

1 Answer 1

1
$\begingroup$

It can be both.

If you input the desired label and predict the next desired label, it's called teacher forcing.

But using only this technique might hurt the performance at test time. So using the actual predicted output from the last time step is also a good idea.

It's possible to do both : For each batch, with X% chance you use teacher forcing, otherwise you don't.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.