LSTM input in Keras

Question

I am confused about the input vector in LSTM model, the data I am using is the text data, e.g. 1,000 sentences. I have two questions about the LSTM input layer:

1.If I would tokenize those sentences into the vectors (we can call it sentence vectors), is there a way in Keras to make sentence vectors given a document? Should be word level, right?

2.The second question is the 3D Tensor type in LSTM. I have 1,000 sentences (samples) and time_step would be 1 if I want to LSTM read one document at each time step, is that correct? The last one is the input dimension, this input dimension is the word dimension (100) in each sentence or how many word observed in each time step (10)?

Thus the LSTM tensor should be (1000, 1, 10) or (1000, 1, 100)

Answer 1

1. For LSTM, the documents should be at word level. Hence, sentence vectors are not that useful for a document but word vectors are. You can use an embedding layer if you want do it though.

2. in the 3D tensor, the first dimension is number of sentences. so 1000 is correct. The second one is the number of time_steps which is the number of words for each sentence. The third one is the word vector dimension. Hence, taking your numerical example, the input dimension of the LSTM will be (1000, 10, 100).