I need to implement and train a Transformer-based model to classify users (binary classification) based on some time-series data. For each user, time-series data are stored as a variable number of fixed-size vectors.

My idea is to train some kind on Transformer. Since it's a classification task, I only need to use the Decoder part. Because the order of events is important, I should use some kind of positional encoding and full (non masked) self-attention.

I now need to implement the architecture and train the model. Do you have suggestions on where I should start from? Also, please let me know if you spot mistakes in my thinking! Thanks.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.