Given a query sentence, we search and find similar sentences in our corpus using transformer-based models for semantic textual similarity.
For one query sentence, we might get 200 similar sentences with scores ranging from 0.95 to 0.55.
For a second query sentence, we might get 200 similar sentences with scores ranging from 0.44 to 0.27.
For a third query sentence, we might only get 100 similar sentences with scores ranging from 0.71 to 0.11.
In all those cases, is there a way to predict where our threshold should be without losing too many relevant sentences? Having a similarity score of
1.0 does not mean that two documents are 2X more similar than if the score was
0.5. Is there a way to determine the
topk (how many of the top scoring sentences we should return) parameter?