What is the difference between nDCG and rank correlation methods?

Question

When do we use one or the other?

My use case:

I want to evaluate a linear space to see how good retrieval results are. I have a set of data X (m x n) and some weights W (m x 1). I want to measure the nearest neighbour retrieval performance on W'X with a ground truth value Y. This is a continuous value, so I can't use simple precision/recall.

If I use rank correlation, I will find the correlation between retrieved Ys and retrieval rank. If I use nDCG, I will use sorted Y to compute IDCG.

I would like to compare this to the correlation value I get when I change Y also. (For example, Y could be head pose angle in one case and age in another)

Answer 1

A central problem in ranking is to design a measure for evaluation of ranking functions. In this paper we study, from a theoretical perspective, the Normalized Discounted Cumulative Gain (NDCG) which is a family of ranking measures widely used in practice. Although there are extensive empirical studies of the NDCG family, little is known about its theoretical properties

NDCG has two advantages compared to many other measures. First, NDCG allows each retrieved document has graded relevance while most traditional ranking measures only allow binary relevance. ... Second, NDCG involves a discount function over the rank while many other measures uniformly weight all positions.

More: Info with Theoretical & Mathematical