I am trying to understand the layers in LSTM for my own implementation using Python. I started with Keras to getting familiarized with the layer flow.

I have tried the below code in Keras and I have the observations as follows

step_size = 3
model = Sequential()
model.add(LSTM(32, input_shape=(2, step_size), return_sequences = True))

And I got the below summary details for this implementation.

enter image description here

I tried to understand the internal structure of these layers and how these layer numbers ( 32, 18,1) mapped with matrix. I got the below matrix size from the weight matrix.

I can not see any relation between the output size and the weight matrix :(

Matrix Order

Layer 1

(3, 128)

(32, 128)


Layer 2

(32, 72)

(18, 72)


Layer 3

(18, 1)


Now I am confused to find out the implementation logic inside the layer. Because I got three weight matrix in each LSTM layer and I can not connect the first LSTM(32) to the next LSTM(18) layer.

It will be thankful if any live implementation or flow diagram exists for this implementation. Thanks.

  • $\begingroup$ What do you mean with connect? For what I can see they are fully connected, for instance the last layer has 19 parameters, that means 18 from the output of the right-layer and its bias. $\endgroup$ – Celius Stingher Aug 13 '19 at 21:35
  • $\begingroup$ Connect means, Layer 1 output shape is (None,2,32) and Layer 2 input should also the same shape. What are the data to be added as input for layer 2. Is the input X[] and state matrix from layer 1? $\endgroup$ – felix Antony Aug 14 '19 at 18:27
  • $\begingroup$ I investigated and found answer. Thank you $\endgroup$ – felix Antony Aug 16 '19 at 14:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.