I use autoregressive model like T5 to tackle the abstractive document summarization task. In my case there are multiple target summarizations for one input document. Is there some related works about how to modify the autoregressive seq-to-seq loss accordingly?

Here is the regular autoregressive loss for the input document $D$ and the target summarization $X$. There is only one target sequence:


However, what if I want to map $D$ to multiple target sequences $\{X^1,...,X^m\}$? I've tried to sum up the loss for each $X^j$ but the test performance is even worse than using only the first target ($X^1$) and ignoring all others.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.