Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Can someone point me to an article which explains how the model training is done in Seq2Seq? I know "Teacher Forcing" is used but what I found so far hasn't been detailed enough. What I am most confused about is where the training happens? The back-propagation goes back to the encoder?
Any insight here would be really appreciated. Thanks!
Backpropagation part ---------------------------
What I am most confused about is where the training happens? The back-propagation goes back to the encoder?
Yes, backpropagation goes from the loss computed with the decoder's output back to the encoder's weights. This is illustrated in this PyTorch tutorial where you see the loss computed with the decoder's output being used to update the encoder's parameters:
loss = 0
(...)
loss += criterion(decoder_output, target_tensor[di])
(...)
loss += criterion(decoder_output, target_tensor[di])
(...)
loss.backward()
encoder_optimizer.step()
decoder_optimizer.step()
Teacher forcing part ---------------------------
I know "Teacher Forcing" is used but what I found so far hasn't been detailed enough.
Teacher forcing implies, most of the time, balancing between 1) feeding the predicted output of the previous time step in the decoder as the decoder's input to predict the next time step 2) feeding the ground truth while ignoring the possibly incorrect predicted output, hence the term "teacher forcing". Now, what is the motivation behind this learning strategy ? It enables to accelerate the seq2seq training. As the same PyTorch tutorial puts it:
“Teacher forcing” is the concept of using the real target outputs as each next input, instead of using the decoder’s guess as the next input. Using teacher forcing causes it to converge faster but when the trained network is exploited, it may exhibit instability.
The tutorial also mentions how one controls how much teacher forcing needs to happen during training with a dedicated hyperparameter:
Turn teacher_forcing_ratio up to use more of it.
More details about teacher forcing and its connection to maximum likelihood are available in Chapter 10 of the excellent Deep Learning book by Goodfellow, Bengio and Courville, section 10.2.1:
We originally motivated teacher forcing as allowing us to avoid back-propagation through time in models that lack hidden-to-hidden connections. Teacher forcing may still be applied to models that have hidden-to-hidden connections as long as they have connections from the output at one time step to values computed in the next time step. As soon as the hidden units become a function of earlier time steps, however, the BPTT algorithm is necessary. Some models may thus be trained with both teacher forcing and BPTT. The disadvantage of strict teacher forcing arises if the network is going to be later used in an closed-loop mode, with the network outputs (or samples from the output distribution) fed back as input. In this case, the fed-back inputs that the network sees during training could be quite different from the kind of inputs that it will see at test time. One way to mitigate this problem is to train with both teacher-forced inputs and free-running inputs, for example by predicting the correct target a number of steps in the future through the unfolded recurrent output-to-input paths. In this way, the network can learn to take into account input conditions (such as those it generates itself in the free-running mode) not seen during training and how to map the state back toward one that will make the network generate proper outputs after a few steps. Another approach (Bengio et al., 2015b) to mitigate the gap between the inputs seen at training time and the inputs seen at test time randomly chooses to use generated values or actual data values as input. This approach exploits a curriculum learning strategy to gradually use more of the generated values as input.
The teacher_forcing_ratio of the PyTorch tutorial previously mentioned thus corresponds to the last approach considered in the cited paragraph, where during training we randomly choose whether to feed the decoder with the ground truth or the predicted output based on a pre-set ratio.
You can check medium page, https://link.medium.com/iolQKIWawX. You can also get more detailed pages on medium itself, shared link includes the references of those.
