How can I specify I want output of which units in LSTM?

Question

I see this code concept(with Keras library) in most code examples of LSTM:

model.add(LSTM(X))
model.add(Dense(Y))

But I don't really know if I have 10 time-steps in input side and need the last time-step in output is the following code true?

model.add(LSTM(10))
model.add(Dense(1))

Or if I have 10 input time-steps and need 10 output time-steps(I mean I need all outputs from all units) is the following code true?

model.add(LSTM(10))
model.add(Dense(10))

How should I specify if I have 1 layer LSTM with 10 units, I need all of 10 outputs? Or I need 1 last output from unit 10? Or the last 5 outputs from unit 5 to 10?

Answer 1

If you use:

LSTM(10 , return_sequences = True)

you will only get as output something of size (batch_size, num_timesteps, 10). The "10" for LSTM unit-size is a hyper-parameter you should be tuning. However, what you want is something the size of (batch_size, num_timesteps, num_features). This is why you want to put a Dense layer after the LSTM layer. That being said, you will also want to add a TimeDistributed wrapper around your Dense layer, so that a Dense layer is applied to every time-step (1):

TimeDistributed(Dense(num_features))

Now the output of the TD dense layer will be of size: (batch_size, num_timesteps, num_features), which I'm assuming is what you want. You will have to set up your Y data to be of the appropriate size for this to work. It will have to be of size (batch_size, num_timesteps, num_features). You mentioned that you are curious to know how to use just part of it. At prediction time, post training, you would call your model.predict and it will return something the size of (batch_size, num_timesteps, num_features). You can just use numpy array slicing to access what it is that you want.