What is the correct way to compute hit rate in recommender systems?

Question

I'm working with the famous Movielens 1M dataset and implemented some simple recommender algorithms. While computing the hit rate, I found that it's very low $(\approx 0.008)$ but the papers seem to report high scores $(\approx 0.5)$. Hence, I think I'm doing something wrong during the evaluation process.

Here's what I am doing: For each movie that the test user hasn't rated, I'm computing a rating using my algorithm. Then I rank these movies and check if the test item occurs in the top 10 list.

After going through many GitHub repos, I found that in some implementations (e.g. SASRec) they randomly sample 100 unrated (by the test user) items and append the test item to it and then they build the top 10 rank list. Using this approach, my hit rate went up but this almost feels like cheating! So, I want to know if this is a common practice or if I failed to understand SASRec's code.

Answer 1

A common way to evaluate machine learning models is performance on unseen data. Thus, "randomly sample 100 unrated (by the test user) items and append the test item to it and then they build the top 10 rank list" is popular.

That process is similar to using precision@k for search engine result pages.