I am trying to create a model starting from the Attention is all you need paper. Specifically I want to setup the Encoder/Decoder architecture to predict time-series.
I would like to implement it in TF/Keras so I gave a look at this tutorial although this is for machine translation. I was not able to find examples of time-series analysis with this architecture with TF/Keras.
However I was able to find some examples in PyTorch as this or this but none of them actually implement de Decoder part. They just implement the Transformer encoder and then a feed forward step to predict the next timestep.
I have also some doubts on how to feed the input to the model as the decoder block starts with a fragment of the target (in case of machine translation this is set to the [START OF SENTENCE] token which I do not think is feasible with time series data.