I'm starting to experiment with neural networks in Google's tensorflow framework, and in particular am working on a model that has an LSTM layer. I am using the RNN tutorial to get me started.
To train my model, I have sequences of data that are several thousand data points long. In the tutorial, the author is processing small sequences of data (short sentences of paragraphs) and concatenates those sequences together, separated by <end of stream>, divides the entire data set equally into batch_size number of chunks and processes those chunks in parallel. I would like to process each of my sequences of data separately from start to finish, but would like to be able to process several of them in parallel, as the rnn_cell objects are set up to do (they process batch_size sequences at onces).
This would be straight forward to do if the sequences were of the same length, but they are not. An rnn_cell.LSTMCell expects the data to come in with a fixed dimensionaltiy, batch_size x input_size, so there's a problem once some of the sequences reach their end and my batch_size wants to decrease.
How can I work around this? I don't think I can 'left' or 'right' justify my data so that they line up without affecting the training.
