0
$\begingroup$

In single-turn dialogue seq2seq models where the goal is to produce a good answer y to a query x, sentences are usually encoded such that x is fed to the encoder, while the decoder is only given a "START" token and the output from the encoder. Often, attention is incorporated into the decoding step by eg concatening the decoder output at time t with the encoder output and performing attention on the resulting vector: attention(y_t, encoder_{out}).

What are common ways to encode multi-turn dialogues? The idea here is that the model can be queried k times (where k is the number of turns in the dialogue) for one conversation and has to keep track of the dialogue context throughout to generate coherent responses.

A simplistic way to do it would be to simply concatenate the dialogue history over all turns and feed it to the encoder, but I am unsure whether different RNNs (eg LSTMs or GRUs) can handle longer sequences and still maintain a high feature quality.

$\endgroup$
6
  • $\begingroup$ Do you mean you want 5 different possible responses given a query and the conversation history? $\endgroup$ Commented Oct 8, 2022 at 18:37
  • $\begingroup$ No, I mean I want a dialogue that lasts 5 turns. I will edit my post to reflect that, thank you for noticing the ambiguity. $\endgroup$ Commented Oct 8, 2022 at 18:40
  • $\begingroup$ Can you just give X as the entire dialog so far? Why wouldn't this not work well? $\endgroup$ Commented Oct 8, 2022 at 18:49
  • $\begingroup$ I am unsure whether different RNNs (eg LSTMs or GRUs) can handle longer sequences and still maintain a high feature quality. Is there some research on when LSTM/GRU performance starts to deteriorate? I also don't know if you can just cram the dialogue history into a vector of fixed size, regardless of how many turns you take. $\endgroup$ Commented Oct 8, 2022 at 18:58
  • 1
    $\begingroup$ A couple options: Fuse the historical dialogue information and the current input statement information in the encoder[Modeling Multi-turn Conversation with Deep Utterance Aggregation][1]. Hierarchical Self-Attention (hrcak.srce.hr/file/379467) $\endgroup$ Commented Oct 8, 2022 at 19:12

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.