I am trying to create a model starting from the Attention is all you need paper. Specifically I want to setup the Encoder/Decoder architecture to predict time-series.

I would like to implement it in TF/Keras so I gave a look at this tutorial although this is for machine translation. I was not able to find examples of time-series analysis with this architecture with TF/Keras.

However I was able to find some examples in PyTorch as this or this but none of them actually implement de Decoder part. They just implement the Transformer encoder and then a feed forward step to predict the next timestep.

I have also some doubts on how to feed the input to the model as the decoder block starts with a fragment of the target (in case of machine translation this is set to the [START OF SENTENCE] token which I do not think is feasible with time series data.

Any recommendation?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy