1
$\begingroup$

When evaluating rankings, Normalized Discounted Cumulative Gain (NDCG) normalizes the score to a [0,1] range by dividing by the ideal score.

What happens if we take the same idea to Precision (Pre) and Recall (Rec) (at k) by dividing the precision and recall by the "ideal" precision and recall?

Obviously, the ideal precision and recall is 1.0, but this only applies while k (the value determining the number of top rankings to consider) is smaller than the number of actual positives (P). Namely, the ideal precision must drop when k exceeds P, since there aren't any more positives to place.

If we consider the formulae:

Pre@k = TP / k

Ideal Pre@k = min(P,k) / k

(where TP is True Positives)

Rec@k = TP / P

Ideal Rec@k = min(k,P) / P

And when we divide the precision and recall by their ideal values, the denominators cancel out and we are left with the same formula: TP / min(P,k)

This behaves like precision for k <= P, and recall for k>=P. Conceptually, they represent the fraction of the ideal precision/recall score, but why would they be the same? Is there some extra meaning I'm unaware of?

Edit:

Would it make sense to average two of these scores if they were calculated from two rankings with differing numbers of positives P? While the average over scores with the same P can still be divided into average precision and average recall halves, using different P's introduces a region where precision and recall is mixed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.