sentence vector to sentence

Question

I have implemented an auto-encoder that takes sentence vectors as input and at decoder the last layer outputs sentence vectors. I would like to convert sentence vectors to sentences. Is there any way to convert sentence vectors to sentences?

class AutoEncoder(nn.Module):

def __init__(self, embedding_dim, hidden_dim):
    
    #Constructor
    super().__init__()          

    self.fc=nn.Linear(embedding_dim,hidden_dim)
    self.act1=nn.ReLU(0.2)
    self.fc1=nn.Linear(hidden_dim,hidden_dim)
    self.act2=nn.ReLU(0.2)
    self.fc2=nn.Linear(hidden_dim,hidden_dim)
    self.act3=nn.ReLU(0.2)
    self.fc3=nn.Linear(hidden_dim,hidden_dim)
    self.act4=nn.ReLU(0.2)
    self.fc4=nn.Linear(hidden_dim,hidden_dim)
    self.act5=nn.ReLU(0.2)
    self.fc5=nn.Linear(hidden_dim,embedding_dim)
def forward(self, X):     
    l1=self.fc(X)
    a1=self.act1(l1)
    l2=self.fc1(a1)
    a2=self.act2(l2)
    l3=self.fc2(a2)
    a3=self.act3(l3)
    l4=self.fc3(a3)
    a4=self.act4(l4)
    l5=self.fc4(a4)
    a5=self.act5(l5)
    l6=self.fc5(a5)
    return l6

Here I have considered word embeddings initially and converted them to sentence vectors by averaging. These vectors are given for auto encoder model.

Let me know whether we have any other approach where we can generate a sentence.

Any kind of reference is helpful.

Answer 1

You might need an encoder-decoder framework that can be implemented using two LSTM/GRU models. The Encoder takes the first phrase in, outputs the last (summary) hidden layer, it is taken by the first Decoder state, alongside BOS token, outputs the second token, uses it to predict the third, and so on until you reach the end of the (response) phrase.