I've already seen several similair questions but I did not understand anything, what is the interest of TimeDistributed? why we need to insert a TimeDistributed layer after LSTM to establish the time series prediction
Ok let's say you have an
LSTM() layer with
return_sequences = True set. That means each LSTM cell in it is outputting its value. The output of the layer is therefore a sequence of outputs, not just the final one. That means the output is a processed time series, with sequential information in it.
Dense() layers to take this information and use it to process the signal further. In particular, you want some Dense architecture to process each of these sequential outputs one by one. By using the
TimeDistributed() wrapper it's like if you're iterating the same
Dense() architecture on this sequential output.
I typically use
TimeDistributed(Dense()) at the bottom of a seq2seq Decoder, in order produce the final output sequence.
I think you mean TimeDistributedDense? Anyway this should clear up you understanding in one way or other.
TimeDistributedDense applies a same Dense (fully-connected) operation to every timestep of a 3D tensor.
The most common scenario for using TimeDistributedDense is using a recurrent NN for tagging task.e.g. POS labeling or slot filling task.
In this kind of task: For each sample, the input is a sequence ($a_1,a_2,a_3,a_4...a_N$) and the output is a sequence ($b_1,b_2,b_3,b_4...b_N$) with the same length. $b_i$ could be viewed as the label of $a_i$. Push $a_1$ into a recurrent neural net to get output $b_1$. Than push $a_2$ and the hidden output of $a_1$ to get $b_2$...
If you want to model this by Keras, you just need to used a TimeDistributedDense after a RNN or LSTM layer(with return_sequence=True) to make the cost function is calculated on all time-step output. If you don't use TimeDistributedDense ans set the return_sequence of RNN=False, then the cost is calculated on the last time-step output and you could only get the last bN.
Also, here is a link for more info -> https://github.com/keras-team/keras/issues/1029