I am trying to create a model starting from the Attention is all you need paper. Specifically I want to setup the Encoder/Decoder architecture to predict time-series.

I would like to implement it in TF/Keras so I gave a look at this tutorial although this is for machine translation. I was not able to find examples of time-series analysis with this architecture with TF/Keras.

However I was able to find some examples in PyTorch as this or this but none of them actually implement de Decoder part. They just implement the Transformer encoder and then a feed forward step to predict the next timestep.

I have also some doubts on how to feed the input to the model as the decoder block starts with a fragment of the target (in case of machine translation this is set to the [START OF SENTENCE] token which I do not think is feasible with time series data.

Any recommendation?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.