0
$\begingroup$

Do I need to specify the input_dim (which means the number of features in one row/sample) after adding the first LSTM layer for the later Dense layers?

I was trying to create an architecture with 2 LSTM layers and 1 Feed-forwarding layer with 200 cells and 1 Feed-forwarding layer with 2 cells. First LSTM layer outputs for every timestamp second LSTM layer outputs for only the last time step so I was wondering If I created that architecture correctly

model.add(LSTM(units = 200, return_sequences = True, input_shape = (WINDOW_SIZE, 9), batch_size = 206))
model.add(LSTM(units = 200, return_sequences = False))
model.add(Dense(units = 200, input_dim = 9))
model.add(Dense(units = 2, input_dim = 9))
model.add(Dense(units = 1, input_dim = 9))
$\endgroup$
1
$\begingroup$

You do not need to specify the input_dim for the later layers, the model can infer the shape of those input layers from the output shape of the previous layer.

In addition, the input_dim values currently specified don't match the output dimensions of the prior layers- for example, the output dimension of the LSTM layer with return_sequences = False will be (200,). The network should fail to compile with an expected input size error with this code.

You can fix this by correcting or removing the input_dim values from the final three layers.

$\endgroup$
  • $\begingroup$ So the output shape of layer is related with the cells in it. For example if I have 50 units that means my output shape will be (50,) right ? $\endgroup$ – Khan9797 Nov 5 '18 at 20:01
  • $\begingroup$ Yes, that's right. $\endgroup$ – Thomas Cleberg Nov 5 '18 at 20:52
  • $\begingroup$ If you find the answer undeful, it helps others find it if you upvote the answer. If it completely answers your question, accepting the answer is how you show that. Welcome to stack exchange! $\endgroup$ – Thomas Cleberg Nov 5 '18 at 23:49
  • $\begingroup$ What is the relation between layer weight matrix size and layer output size? Is there any relation? $\endgroup$ – felix Antony Aug 12 at 20:44
  • $\begingroup$ The layer weight matrix is a function of how many units there are in the layer, how many outputs each unit has and how many weights each unit in that layer has. For most conventional layers, this means 1 weight per unit in the layer, 1 output per unit in the layer and they would be equal. The extra information I provided here is only meant to illustrate that it's not always the case, and that you architecture could potentially change the answer. $\endgroup$ – Thomas Cleberg Aug 14 at 12:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.