I read about them in Keras documentation and other websites, but I couldn't exactly understand what exactly they do and how should we use them in designing many-to-many or encoder-decoder LSTM networks?

I saw them used in the solution of this problem here.

model = Sequential()  
model.add(LSTM(input_dim=1, output_dim=hidden_neurons, return_sequences=False))  
model.add(LSTM(output_dim=hidden_neurons, return_sequences=True))  
model.compile(loss='mean_squared_error', optimizer='rmsprop', metrics=['accuracy'])
  • $\begingroup$ Here in this example, RepeatVector() is taking the output of the first LSTM and repeats it 10 times. That gives the second LSTM a sequences of lenght=10. But I don't get the intuition behind repeating a vector before giving it to another LSTM ? $\endgroup$ Apr 2, 2020 at 12:02
  • $\begingroup$ found a good tutorial that explains everything about encoder-decoder architecture including RepeatVector() and TimeDistributed() here: machinelearningmastery.com/… $\endgroup$ Apr 3, 2020 at 8:27

2 Answers 2



According to the docs :

Repeats the input n times.

They have also provided an example :

model = Sequential()
model.add(Dense(32, input_dim=32))
# now: model.output_shape == (None, 32)
# note: `None` is the batch dimension
# now: model.output_shape == (None, 3, 32)

In the above example, the RepeatVector layer repeats the incoming inputs a specific number of time. The shape of the input in the above example was ( 32 , ). But the output shape of the RepeatVector was ( 3 , 32 ), since the inputs were repeated 3 times.


According to the docs :

This wrapper allows to apply a layer to every temporal slice of an input. The input should be at least 3D, and the dimension of index one will be considered to be the temporal dimension.

You can refer to the example at their website.

TimeDistributed layer applies a specific layer such as Dense to every sample it receives as an input. Suppose the input size is ( 13 , 10 , 6 ). Now, I need to apply a Dense layer to every slice of shape ( 10 , 6 ). Then I would wrap the Dense layer in a TimeDistributed layer.

model.add( TimeDistributed( Dense( 12 , input_shape=( 10 , 6 ) )) )

The output shape of such a layer would be ( 13 , 10 , 12 ). Hence, the operation of the Dense layer was applied to each temporal slice as mentioned.

  • 2
    $\begingroup$ and when should they be used? $\endgroup$
    – Ben
    Nov 27, 2019 at 15:01
  • 1
    $\begingroup$ @Shubham I think that the question is about "why" are they using RepeatVector() and not just one LSTM with return_sequence=True. I don't get the intuition behind repeating the output of the first LSTM and feeding it to the second one. $\endgroup$ Apr 2, 2020 at 12:00
  • $\begingroup$ I could be wrong, but I think the OP was asking about these layers in a practical sense (he mentioned autoencoders and sequence to sequence), which you didn't address. Can you clarify your answer to include the WHY they'd be used in these contexts? It's pretty obvious what RepeatVector() does, but why would you want to use it for a encoder-decoder model? $\endgroup$ Dec 6, 2020 at 18:02

For encoder-decoder, your input is squashed into a single feature vector, if you want your output to regenerate the same dimension as the original input, you are "artificially" converting this feature tensor from 1D into 2D by replicating it using RepeatVector().

  • $\begingroup$ this was the answer I was looking for $\endgroup$ Jan 13, 2021 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.