# Why use two LSTM layers one after another?

In the example on the Keras site, seq2seq_translate.py on line 189, there is a LSTM layer after another (the first with return_sequences=True), but another example lstm_seq2seq.py which does the same thing but letter-by-letter uses only one LSTM in the encoder. My code looks like:

encoder = LSTM(latent_dim, return_sequences=True)(encoder_inputs)
encoder_outputs, state_h, state_c = LSTM(latent_dim, return_state=True)(encoder)


My question is why does the word-by-word version use two LSTM layers? And why is the return_sequences used?

## 1 Answer

About your first question: It is because word-by-word NLP model is more complicated than letter-by-letter one, so it needs a more complex network (more hidden units) to be modeled suitably.

About your second question: When you want to use two-staged LSTMs, the hidden sequence of first LSTM must be used as input of the second LSTM and the return_sequences option is used to do this.

• Thanks @pythinker - I added some code to my question - so I've got the return_sequences and return_state around the right way? – Nic Cottrell Aug 12 '18 at 20:18
• @NicCottrell Yes, that is right, but note that inclusion of return_state is optional. – pythinker Aug 12 '18 at 21:36
• return_state=False by default though, so isn't it required for state_h, state_c to be returned? – Nic Cottrell Aug 13 '18 at 7:25
• @NicCottrell It is absolutely required for state_h and state_c to be returned but not for encoder_outputs. – pythinker Aug 13 '18 at 15:28