I am confused about the input vector in LSTM model, the data I am using is the text data, e.g. 1,000 sentences. I have two questions about the LSTM input layer:

1.If I would tokenize those sentences into the vectors (we can call it sentence vectors), is there a way in Keras to make sentence vectors given a document? Should be word level, right?

2.The second question is the 3D Tensor type in LSTM. I have 1,000 sentences (samples) and time_step would be 1 if I want to LSTM read one document at each time step, is that correct? The last one is the input dimension, this input dimension is the word dimension (100) in each sentence or how many word observed in each time step (10)?

Thus the LSTM tensor should be (1000, 1, 10) or (1000, 1, 100)

  1. For LSTM, the documents should be at word level. Hence, sentence vectors are not that useful for a document but word vectors are. You can use an embedding layer if you want do it though.

  2. in the 3D tensor, the first dimension is number of sentences. so 1000 is correct. The second one is the number of time_steps which is the number of words for each sentence. The third one is the word vector dimension. Hence, taking your numerical example, the input dimension of the LSTM will be (1000, 10, 100).

| improve this answer | |
  • $\begingroup$ I want to make sure the time steps is the number of observed words for each sentence, right? $\endgroup$ – Mike Oct 13 '16 at 19:29
  • $\begingroup$ Yes @Mike, it is the number of words in each sentence. $\endgroup$ – Hima Varsha Oct 14 '16 at 4:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.