precision@k and recall@k

Question

Normally, I am familiar with precision and recall evaluation metrics but as you know recall@k and precision@k are different things and used in ranking evaluations especially recommendation systems.

I checked many sources, I understood everything I could not understand a point.

One more thing,

Every source is different between each other in terms of calculation ( 1 , 2, 3, 4 )

let's get this example

I'll give you an example so you can explain to me

Let's say we have 5 users. We are trying to give location recommendations for the next visit to each user. We are analyzing users' historical check-in data and we are giving recommendation for the next visit.

User 1 is visiting: Museum1, Park1, Night Club, "?" (What is next)

We are trying to find the next visiting locations. Let's say our ground truth "Restaurant"

How can I calculate precision@5 and recall@5?

Extra: This youtube video is explaining very good (go to 51:45 on video)

What is 5-6 relevant item means? If we are giving recommendation it should be just 1 item that is gonna be relevant for the user. They are trying to make a movie recommendation but they are saying we have 5 relevant movies. What is that mean?

Answer 1

A quick answer.

Note: Checking the references I could access fully, there are no discrepancies between the definitions as long as the terms are translated correctly.

Some definitions:

Relevant items: Are items the user(s) have themselves established as relevant with their actions.

Recommended items: Are items which the recomendation system predicts will be relevant to the user(s).

Concerning your example:

Let’s understand the definitions of recall@k and precision@k, assume we are providing $5$ recommendations in this order — 1 0 1 0 1, where 1 represents relevant and 0 irrelevant. So the precision@k at different values of k will be precision@3 is $2/3$, precision@4 is $2/4$, and precision@5 is $3/5$. The recall@k would be, recall@3 is $2/3$, recall@4 is $2/3$, and recall@5 is $3/3$.