I've already seen several similair questions but I did not understand anything, what is the interest of TimeDistributed? why we need to insert a TimeDistributed layer after LSTM to establish the time series prediction


I think you mean TimeDistributedDense? Anyway this should clear up you understanding in one way or other.

TimeDistributedDense applies a same Dense (fully-connected) operation to every timestep of a 3D tensor.

The most common scenario for using TimeDistributedDense is using a recurrent NN for tagging task.e.g. POS labeling or slot filling task.

In this kind of task: For each sample, the input is a sequence (a1,a2,a3,a4...aN) and the output is a sequence (b1,b2,b3,b4...bN) with the same length. bi could be viewed as the label of ai. Push a1 into a recurrent nn to get output b1. Than push a2 and the hidden output of a1 to get b2...

If you want to model this by Keras, you just need to used a TimeDistributedDense after a RNN or LSTM layer(with return_sequence=True) to make the cost function is calculated on all time-step output. If you don't use TimeDistributedDense ans set the return_sequence of RNN=False, then the cost is calculated on the last time-step output and you could only get the last bN.

Also, here is a link for more info -> https://github.com/keras-team/keras/issues/1029


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.