Give Variable Length input to LSTM

Question

My input data consist of list of list. Both list have dynamic length for every example like below.

X[0] = [[0, 1, 3, 5, 8, 26], [3, 4, 5, 7, 9, 15, 26, 30, 38, 39, 40]]
X[1] = [[1, 2, 4, 16, 19, 29, 48]]
..

My target data consist of dynamic length list like below:

y[0] = [5, 7, 8, 12, 15, 17, 29]
y[1] = [2, 4, 6, 8, 10, 11, 16, 18, 19, 29, 30, 33, 35]
..

I try to train LSTM model with given input X, predict Y.

If I concat each X list to one list and make padding operation LSTM accepts that data type. My example code is below:

X_train = sequence.pad_sequences(X_train, maxlen=padding_size)
X_test = sequence.pad_sequences(X_test, maxlen=padding_size)

model = Sequential()
model.add(Embedding(50, 10, input_length=X_train.shape[1], mask_zero=True))

if isBidirectional:
    model.add(Bidirectional(LSTM(lstm_layer_number)))
else:
    model.add(LSTM(lstm_layer_number))

if isDropout:
    model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) # binary crossentropy yields the best results

The problem is that I don't want to give input as a fixed size vector. Instead of given fixed size vector, I want to give input as a dynamic size and list by list.

How can I give these input as sequence(list by list) to LSTM ?

Any help is appreciated.

Answer 1

You can try padding the inputs with zero that is of lesser in length.

Answer 2

Most of time the simplest method is zero-padding. However, if there is a lot of short sequence and a few very long sequence you could have a lot of useless computation of 0 and this may be worth to optimise it like described in appendix B of https://www.researchgate.net/publication/363027076_Use_of_Deep_Learning_to_Detect_the_Maternal_Heart_Rate_and_False_Signals_on_Fetal_Heart_Rate_Recordings

The idea is to concatenate small sequences in a batch to make longer sequences and to force the state to be resetted between each sequence.

An example with tensorflow on GRU is at https://github.com/utsb-fmm/FHRMA/blob/master/FS%20training%20python%20sources/2_Training_FSMHR.ipynb

Look at the reseter function