I am currently reading Semantic Product Search paper published by Amazon. They are using two evaluation subtasks matching and ranking. In matching, they tune the model hyperparameters to maximize Recall@100 and Mean Average Precision (MAP).
According to Introduction to Information Retrieval, Precision (P) is the fraction of retrieved documents that are relevant:
Recall (R) is the fraction of relevant documents that are retrieved:
I want to know how to come up with ground truth(relevancy label) if it's not available? In other words, if I want to calculate precision or recall for the Semantic product search and if we don't have relevancy label available for input product query. In that case, how researchers calculate precision and recall? or how do they generate it?