1
$\begingroup$

I am using a LSA/TF-IDF/BM25/Ensemble models for text search and finally calculating similarity score to rank my search. I would like to decide a threshold value for the score, below which I would not like to display anything. Eg: If my similarity score is less than 0.7, I would like to return "No Result Found".

I know there wont be any specific value that I can use here, but I would appreciate suggestions on how I can find that value?

My Thoughts: Idea 1: I can calculate the similarity scores for all past searches and try to find average of top 10 scores for each search, and take a final mean of all those value (Not sure if it would be a good idea).

Idea 2: Deploy model to production with top 15-20 search results and wait to collect users click results, so one insight after collecting result for a month could be, 95% of the time user does not go to any results having score less than 0.60 or something.

$\endgroup$
2
  • 1
    $\begingroup$ There's no perfect answer, your ideas seem good to me. I would say that the 2nd one is certainly better (it's practically a kind of evaluation) but it's more complex to apply. In case it helps, see related question. $\endgroup$
    – Erwan
    Sep 25, 2022 at 21:25
  • $\begingroup$ @Erwan Thank You for your response, and indeed the related question you shared is helpful. $\endgroup$ Sep 26, 2022 at 6:43

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.