Since CTC loss is also intended to deal with seq2seq mappings, I wonder about how the two loss functions compare. I went through an excellent explanation of CTC loss, finding out that the target sequence is restricted not to be longer than the input sequence whereas this restriction does not exist for seq2seq loss.
Some questions I have (ranked by relevance):
- Are bidirectional LSTMs suited for seq2seq loss? I have only found b-LSTM implementations with CTC or even simpler loss functions. There is a post from 2016 saying that this was not possible at that time. Any updates?
- Are there conceptual differences apart from the many-to-one restriction in CTC loss that seq2seq does not have? E.g. does the seq2seq loss make a similar conditional independence assumption like the CTC loss?
- Do the loss functions have preferences regarding the optimizer (RMSProp, ADAM, Momentum etc.)
- Do the loss functions work better for particular sets of tasks?
Theoretical arguments and practical experiences are both appreciated.