Keras- LSTM answers different size

Question

Imagine situation, input is some text file and information are spread according to rows. I want to use rows as input of model.

Model is planned as LSTM with softmax as output layer. My problem is the output; I want to classify text file to some categories. So I imagine this like, I open text file, do necessary preprocessing and row by row feed into the model and after I have applied all the rows to the file, I tell softmax which category is right for this document. But I don't know how to make softmax layer "wait" until last row of file is processed by LSTM layers.

For example, I have file with 1000 rows and every row has 5 features. So my dataset is [1000x5] size but I don't have 1000 y I have 1 y which told me category of this document. Documents can be different size so I can't use whole document as input. I also don't see as option make y size 1000 where everything is category index.

When I think about this seems to me, when person implement LSTM from scratch, is possible to make LSTM run(save every state) and after several iterations perform softmax and than apply BTT or more suitable algorithm and perform weights update.

Every LSTM keras tutorial which I saw uses y which is the same shape as number of samples. Maybe I didn't see right ones.

Answer 1

Your dataset is not [1000x5], one of your documents is that size. LSTMs can deal with variable sized input directly or by padding (this is the case with Keras). In Keras you can use the parameter return_sequences as False in your LSTM, which means only after the last part of your sequence has been parsed it will transform it with a Softmax and run the classification. What I would do is pad your documents to a certain size, for example 1000 (though this is high for LSTMs) so that you will have a [n x 1000 x 5] feature set and a [n] target set. You can feed this to your model now.