1
$\begingroup$

So I have 82 different sets of data, each with varying length where each point has one feature and a label (0 or 1).

I'm trying to use Keras LSTM to be able to predict the class of a point depending on the previous values before it.

Currently I've padded each set to be of the same length and plan on using a masking layer.

What I'm confused with after reading many examples is how I should reshape my data and supply the correct input shape to the model.

If each set is of length 55, and I want to consider the previous 5 (just throwing a number out there) points to predict the class at each step, how would I do that?

Should I manually use a moving window to break each set into arrays of length 5, then have input_shape=(5, 11, 1) since 55/5 = 11??

Is this the best approach? How would this work for the remaining 80+ sets?

Just to clarify: say one of the 82 sets looks like this [1.1, 4.8, ... 2.1] (length of 55) I want the model to predict the class of each point [0, 1, ... 1] (x55) except for the padded points (-1).

$\endgroup$
1
$\begingroup$

I don't see much point in doing sliding windows with LSTM. The whole idea of this layer is to capture long-term dependencies and you're just throwing it away with the 5 step window limit.


The easiest for this case would be using a Conv1D with kernel_size=5 instead. Then the windows would go automatically with the kernel size. (But only one layer, if you add more, you will capture more steps). Other possibilities are 2 layers with kernel size = 3, or maybe 3 layers with sizes 3, 2, 2.


If you do want to use windows with LSTM, you will have to organize the data manually. This means you will loop your data and get segments of length 5 and treat each segment as an individual sequence.

In this case your input shape will be (5,1) and you will have far more than 82 samples. On the other hand, if all your sets are longer than length 5, you will need no padding at all.

Example loop:

originalData = load_a_list_of_samples()
windowData = []
for sample in originalData:
    L = len(sample) #number of time steps
    for segment in range(L - 5 + 1):
        windowData.append(sample[segment:segment+5])

windowData = np.array(windowData)

My suggestion though, unless you have this requirement for some special reason, is don't use windows at all. Let the LSTM do their job and capture long term dependencies.

Pad the data as intended, use the Masking layer and place the LSTM with return_sequences=True.

The results will be like (showing only the length dimension)

inputs = [step1 , step2 , step3 , step4 , step5 , step6 , step7 ]     
outpus = [class1, class2, class3, class4, class5, class6, class7]

Make sure you mask your loss function as well, as I'm not sure Keras will do the masking job correctly there.

$\endgroup$
2
  • $\begingroup$ Okay so I've implemented it with a sliding window of size 5, where each sequence is X = [step1, step2, step3, step4, step5], y = [step5_label]. Lets say I pad the sequences and use return_sequences=True to get the full output. Then my supervised labels should also be of the same dimension? Would I end with a Dense(1) layer for binary classification? What would be my layers after the LSTM since return sequences will output many dimensions? Thanks $\endgroup$ Dec 3 '19 at 17:08
  • $\begingroup$ There is no full sequence in this case. You must predict each window and concatenate the results if you want a full sequence of labels. There is no padding for sliding windows too. $\endgroup$ Dec 3 '19 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.