0
$\begingroup$

I need to implement and train a Transformer-based model to classify users (binary classification) based on some time-series data. For each user, time-series data are stored as a variable number of fixed-size vectors.

My idea is to train some kind on Transformer. Since it's a classification task, I only need to use the Decoder part. Because the order of events is important, I should use some kind of positional encoding and full (non masked) self-attention.

I now need to implement the architecture and train the model. Do you have suggestions on where I should start from? Also, please let me know if you spot mistakes in my thinking! Thanks.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.