2
$\begingroup$

I see this code concept(with Keras library) in most code examples of LSTM:

model.add(LSTM(X))
model.add(Dense(Y))

But I don't really know if I have 10 time-steps in input side and need the last time-step in output is the following code true?

model.add(LSTM(10))
model.add(Dense(1))

Or if I have 10 input time-steps and need 10 output time-steps(I mean I need all outputs from all units) is the following code true?

model.add(LSTM(10))
model.add(Dense(10))

How should I specify if I have 1 layer LSTM with 10 units, I need all of 10 outputs? Or I need 1 last output from unit 10? Or the last 5 outputs from unit 5 to 10?

Improve this question
$\endgroup$
  • 1
    $\begingroup$ If you need the output as sequences, don't add the Dense layer. Instead do LSTM( X , return_sequences=True ). This will return all the sequences. $\endgroup$ – Shubham Panchal Mar 1 '19 at 12:19
  • $\begingroup$ @ShubhamPanchal: What if I only need the 5 or 3 or 1 last output of the sequence? How can I specify this? $\endgroup$ – user145959 Mar 1 '19 at 12:32
  • $\begingroup$ Are all your features numeric, and you want the next time-step prediction for each time-step of the 10? As far as I know, there isnt a way to only get part of the outputs only. You can just return sequences=true and only use part of it, however. $\endgroup$ – kylec123 Mar 1 '19 at 15:21
  • $\begingroup$ @kylec123: May you put a simple code for that? I don't know is it necessary to put a Dense() layer after LSTM(10 , return_sequences = True) layer to get the sequences or not? I also don't know how to get the complete sequence and split some part of it? $\endgroup$ – user145959 Mar 1 '19 at 15:54
  • $\begingroup$ @kylec123: Yes, all of my data is numeric data. $\endgroup$ – user145959 Mar 1 '19 at 15:55
1
$\begingroup$

If you use:

LSTM(10 , return_sequences = True)

you will only get as output something of size (batch_size, num_timesteps, 10). The "10" for LSTM unit-size is a hyper-parameter you should be tuning. However, what you want is something the size of (batch_size, num_timesteps, num_features). This is why you want to put a Dense layer after the LSTM layer. That being said, you will also want to add a TimeDistributed wrapper around your Dense layer, so that a Dense layer is applied to every time-step (1):

TimeDistributed(Dense(num_features))

Now the output of the TD dense layer will be of size: (batch_size, num_timesteps, num_features), which I'm assuming is what you want. You will have to set up your Y data to be of the appropriate size for this to work. It will have to be of size (batch_size, num_timesteps, num_features). You mentioned that you are curious to know how to use just part of it. At prediction time, post training, you would call your model.predict and it will return something the size of (batch_size, num_timesteps, num_features). You can just use numpy array slicing to access what it is that you want.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.