2
$\begingroup$

I am new in making recommendation systems . I am using the surpriselib library to evaluate my recommendations. All the Accuracy Metrics are well supported in this library. But I also want to compute the Hit Rate of my top n recommender system.

I know the formula for hit rate is:

(no items users have already purchased)/(no of users)

But this does not makes sense to me because to train and test the user vs item ratings I have only those users in my dataset which have already rated all the items that are in the dataset. (**After all this is what enables me to measure accuracy of my rating predictions **)

This means I will get a HIT RATE OF 100% everytime !!!

Is my theory wrong if not what can I do to properly measure hit rate I know there is one method called Leave one out cross validation which can help me in this case . If that's the answer how do they work with top n recommender systems?

$\endgroup$

1 Answer 1

0
$\begingroup$

Here is my procedure for computation of the "Hit Rate with leave-one-out cross validation":

  1. leave only one actual user-item interaction (this can be rating, purchase or whatever you are dealing with in your data) out of the train dataset (this gives you a total of U "unseen" interactions provided that you have at least one interaction per user);
  2. Generate top n recommendations for all user with whatever recommendation system you are using;
  3. if a user actually interacted with the recommended item (is the left-out" item was among the top n recommendations for that particular user) => that's a hit Hit rate = sum of all the hits across the users (this can be at most U since you can have at most 1 hit per user) DIVIDED BY the number of users (U).
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.