0
$\begingroup$

I want to perform binary classification on a sequential data. I want to leverage the PyTorch nn.TransformerEncoder module, is there a way I can do this? If not, are Transformers only used NLP? If yes, how can I format the inputs and what tutorial can help me do this as quick as possible

$\endgroup$
1
  • 2
    $\begingroup$ Yes, you can do that. Look into some kaggle comps. $\endgroup$
    – Aditya
    Jan 4 at 18:20
0
$\begingroup$

Transformers can be used for classification tasks. I found a good tutorial where they used a BERT Transformer for the encoding and a Convolutional Neural Network for a sentiment analysis.

You can also fine-tune a whole Transformer for the classification but this is usually pretty intense when it comes to the training and you definitely need a GPU.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.