I have a list of sentences called X. Each sentence is associated with a sentence in a list Y. I'm trying to use BERT to create a model that, given a sentence in X, will predict which of the sentences in Y is associated with that X.
I've already trained a fine-tuned model to produce a vector that is, in theory, very close in cosine similarity to the associated sentence in Y. I got a cosine-similarity loss of like -.96. However, when I try to use this model in another model to predict which of the sentences in Y is linked to a given sentence in X, I get a model with low training and validation accuracy.
What techniques or other algorithms can I use?
