0
$\begingroup$

I'm using BERT to encode sentences. The sentences I'm encoding are quite similar, meaning they all belong to the same overall topic. Therefor, I am using another parameter for measuring similarity. So, when I want to add even more similarity to BERT's encodings, I simply pad a vector of size 100 filled with value 1 to the beginning of BERT's vectors. This makes them more similar when cluster / doing similarity measurement.

Now this i somewhat of a hack, and I probably should except it to perform on all sentence input types. Do any of you have an idea of how to achieve the same, in a more correctly / scalable way.

I've tried finetuning BERT, though this simply takes to long and uses to many resources for my task.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.