2
$\begingroup$

I am trying to build an encoder-decoder model for a text style transfer problem. The problem is I don't have parallel data between the two styles so I need to train the model in an unsupervised setting.

Some papers I have seen use an auto-encoder to train the encoder and decoder components separately. By setting the problem as an auto-encoder, they can train the decoder by passing the target sequence (equal to the input sequence) into the decoder. (Here are some examples, https://arxiv.org/pdf/1711.06861.pdf, https://arxiv.org/pdf/1804.04003.pdf)

Instead of an auto-encoder, I would like to know if it's possible to train a decoder by feeding its predictions at time, t-1, into the input at time-step t. I would pass the generated output into a classifier to check the style and to obtain a training signal. Is this sensible and what are the pros / cons of doing so? Thanks.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

I would like to know if it's possible to train a decoder by feeding its predictions at time, t-1, into the input at time-step t.

Yes, it is possible to do it.

But I don't see why you would do it.

Υou will have accumulated error propagated and amplified in every new prediction, making your prediction to diverge from the ground truth sooner or later.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your answer. I understand your point about why the error would be propagated. The reason I am considering it is because I don't have a ground truth to input to the decoder. $\endgroup$
    – Physbox
    Commented Jul 8, 2018 at 22:39
  • $\begingroup$ it is like an open-loop control system :) $\endgroup$
    – pcko1
    Commented Jul 8, 2018 at 22:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.