I am trying to fine-tune a Bert model for sentiment analysis. Instead of one sentence, my inputs are documents (including several sentences) and I am not removing dots. I was wondering if is it okay to use just the embedding of the first token in such cases. If not, what should I do?
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Yes, it's perfectly fine to fine-tune BERT on sequences comprised of more than one sentence, and the standard way of using BERT for text classification is with the ouput vector at the first position.
However, take into account that the maximum length of BERT's input sequences is 512 tokens, so your documents should be short enough to fit in that.
