1
$\begingroup$

I have implemented an auto-encoder that takes sentence vectors as input and at decoder the last layer outputs sentence vectors. I would like to convert sentence vectors to sentences. Is there any way to convert sentence vectors to sentences?

class AutoEncoder(nn.Module):

def __init__(self, embedding_dim, hidden_dim):
    
    #Constructor
    super().__init__()          

    self.fc=nn.Linear(embedding_dim,hidden_dim)
    self.act1=nn.ReLU(0.2)
    self.fc1=nn.Linear(hidden_dim,hidden_dim)
    self.act2=nn.ReLU(0.2)
    self.fc2=nn.Linear(hidden_dim,hidden_dim)
    self.act3=nn.ReLU(0.2)
    self.fc3=nn.Linear(hidden_dim,hidden_dim)
    self.act4=nn.ReLU(0.2)
    self.fc4=nn.Linear(hidden_dim,hidden_dim)
    self.act5=nn.ReLU(0.2)
    self.fc5=nn.Linear(hidden_dim,embedding_dim)
def forward(self, X):     
    l1=self.fc(X)
    a1=self.act1(l1)
    l2=self.fc1(a1)
    a2=self.act2(l2)
    l3=self.fc2(a2)
    a3=self.act3(l3)
    l4=self.fc3(a3)
    a4=self.act4(l4)
    l5=self.fc4(a4)
    a5=self.act5(l5)
    l6=self.fc5(a5)
    return l6

Here I have considered word embeddings initially and converted them to sentence vectors by averaging. These vectors are given for auto encoder model.

Let me know whether we have any other approach where we can generate a sentence.

Any kind of reference is helpful.

$\endgroup$
2
  • $\begingroup$ You could do it with an LSTM or any other recurrent network. However, averaging the vectors means that you are losing the original ordering of words, so you may consider obtaining the sentence vector in a different way. $\endgroup$
    – noe
    Jan 11 at 17:55
  • $\begingroup$ Do we have any approach to convert sentence vectors back to sentence $\endgroup$ Jan 12 at 3:45
1
$\begingroup$

You might need an encoder-decoder framework that can be implemented using two LSTM/GRU models. The Encoder takes the first phrase in, outputs the last (summary) hidden layer, it is taken by the first Decoder state, alongside BOS token, outputs the second token, uses it to predict the third, and so on until you reach the end of the (response) phrase.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.