0
$\begingroup$

I am trying to fine-tune a Bert model for sentiment analysis. Instead of one sentence, my inputs are documents (including several sentences) and I am not removing dots. I was wondering if is it okay to use just the embedding of the first token in such cases. If not, what should I do?

$\endgroup$
0

1 Answer 1

0
$\begingroup$

Yes, it's perfectly fine to fine-tune BERT on sequences comprised of more than one sentence, and the standard way of using BERT for text classification is with the ouput vector at the first position.

However, take into account that the maximum length of BERT's input sequences is 512 tokens, so your documents should be short enough to fit in that.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.