I'm using BERT to encode sentences. The sentences I'm encoding are quite similar, meaning they all belong to the same overall topic. Therefor, I am using another parameter for measuring similarity. So, when I want to add even more similarity to BERT's encodings, I simply pad a vector of size 100 filled with value 1 to the beginning of BERT's vectors. This makes them more similar when cluster / doing similarity measurement.

Now this i somewhat of a hack, and I probably should except it to perform on all sentence input types. Do any of you have an idea of how to achieve the same, in a more correctly / scalable way.

I've tried finetuning BERT, though this simply takes to long and uses to many resources for my task.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.