0
$\begingroup$

In general we use word2vec for word embedding in seq2seq model, is it possible to add the document vector from Doc2vec with the input words , I mean using the tag of the document as a word and its vector for emending : the picture will explain mu point a view

default seq2seq enter image description here

My opinion enter image description here

the doc_tag is followed by its original words so will that improve my seq2seq model ( logically ) ?

$\endgroup$
  • $\begingroup$ I was also thinking on the same front. Do you have any progress with this approach ? I would be happy to know if doc2vec helped the training or not? Thanks again. The question I feel is valid as Tensorflow has embedding_attention_seq2seq model which actually does create embeddings but not in the way doc2vec creates. Thanks again. $\endgroup$ – Prateek Bhatt May 24 '17 at 9:20
0
$\begingroup$

The purpose of the encoder (green part) is to determine this document vector, so if you want to provide it yourself use the decoder and feed the embedding as the initial state.

I suppose you could use your embedding as a prior for the encoder output. One alternative to your suggestion, which might work, is to use the embeddings as an MSE regularizer to the encoder output.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.