I am an beginner in text generation and and deep-learing but I like to get in touch with it. Currently I am learing about LSTM networks and VAEs for text generation. I would like to read a sequence at once and output another sequence. What I learned from this post is that the input shape should be 3D. Since my text data is 2D I want to reshape it. But this is now the confusing part. As I have training data with the shape of
(nb_sequences, max_seq_lenth) = (274, 95). As the post pointed out, it would be important to reshape the data to a format of input = (nb_sequence, nb_timestep, nb_feature) with nb_sequence being the number of sequences (274), nb_timestep the length of the sequence (95) and nb_feature the number of features describing a sequence at a timestep, right?
I have two questions here:
1). How to reshape properly? My vocab size is 359 which I imagined (obviously wrong) to be the number of the features. --> input = (274, 95, 359). This is not possible since an array of 274*95 = 26030 can't be reshaped into (274, 95, 359).
If I take the number of diffrent words my sequences are containing, wouldn't it lead to some erros since sentences do not consist out of the same words naturally? Meaning that I would have a variable feature number with each sequence?
2). How would one realize the problem I am facing? I read about one-hot encoding the sequences. This might solve the problem I have regarding question 1), but as far as I understand this would lead me to a quite inefficient way of predicting sequences as I want to handle more data. Is there any suggestion how to do it in a effient way?