1
$\begingroup$

I am trying to calculate coverage metrics for a recommender system that I have designed. This blog post talks about how to do it. I had some difficulties in understanding the same. It says that These metrics will be for user u of U users over n items out of N potential items in a recommendation list L. Each item has a content vector C of length c ... My difficulty arises in understanding N and U. Is U the total number of users? Secondly, what does N potential items mean? Is it the list of all items that are present or is it a subset of all the items present?

I am using the Python LightFM package to generate the recommendations

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

Right, the capital letters denote the total available. In that blog post: U means all users, N means all items but in other places is usually written I, and L means all top-n recommendation lists. "Top-n" means that the recommender system outputs a ranked list of n items, so if you had 1000 users all getting a Top-10 list, you'd have L length of 1000*10.

I suggest you read Ge, Mouzhi, Carla Delgado-Battenfeld, and Dietmar Jannach. "Beyond accuracy: evaluating recommender systems by coverage and serendipity." Proceedings of the fourth ACM conference on Recommender systems. ACM, 2010, at http://ls13-www.cs.tu-dortmund.de/homepage/publications/jannach/Conference_RECSYS10b.pdf instead. It's a more clearly written definition of metrics and uses more traditional recommender system notation.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ The ACM paper you refer to refers to the jth recommendation time. Won't the recommendations be the same at every instance? It can only change unless there is a change in the model. So this would mean that catalog coverage would be across different models, or something of that sort. $\endgroup$
    – pseudomonas
    Commented Nov 7, 2017 at 4:27
  • 2
    $\begingroup$ In that paper, their "Catalog coverage" metric is defined as the count of the union of all Top-N recommendation lists for all users, divided by the count of all items that could have been recommended. I also am a bit confused by their j and N notation. An example: you have 5 items in your database: A-E. You have 3 users who got these Top-2 lists: [A,B], [B,C], and [A,C]. The union of those lists is [A,B,C] which is 3 items. 3 divided by 5 items that could have been recommended is 3/5 or 0.6. Catalog coverage is therefore 0.6. $\endgroup$
    – Dan Jarratt
    Commented Nov 7, 2017 at 19:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.