I need to implement and train a Transformer-based model to classify users (binary classification) based on some time-series data. For each user, time-series data are stored as a variable number of fixed-size vectors.
My idea is to train some kind on Transformer. Since it's a classification task, I only need to use the Decoder part. Because the order of events is important, I should use some kind of positional encoding and full (non masked) self-attention.
I now need to implement the architecture and train the model. Do you have suggestions on where I should start from? Also, please let me know if you spot mistakes in my thinking! Thanks.
