sklearn.metrics.average_precision_score getting different answers for same data but different formats

Question

I was trying to learn how average precision (AP) is calculated and implemented in scikit-learn. I have read the documentation, but I don't think I fully understand it yet.

Consider the following two snippet:

import numpy as np
from sklearn.metrics import average_precision_score
y_true = np.array([0, 1, 0])
y_scores = np.array([0.4, 0.4, 0.8])
average_precision_score(y_true, y_scores) # 0.3333333333333333

and

y_true = np.array([[1,0], [0,1],[1,0]])
y_scores = np.array([[0.6,0.4], [0.6,0.4], [0.2,0.8]])
average_precision_score(y_true, y_scores) # 0.45833333333333326

From what I understand, these are the same data but formatted in different ways. The first is only showing the true labels and predicted scores for the positive class, while the second is giving information on both classes.

But why are they giving different answers? In particular, how are these two results calculated? And which one is correct? I was reading this post, but I didn't understand how that precision-recall table in the answer is constructed. Can anyone can go through a similar calculation for my example?

Answer 1

By explicitly giving both classes, sklearn computes the average precision for each class. Then we need to look at the average parameter: the default is macro:

Calculate metrics for each label, and find their unweighted mean...

If you switch the parameter to None, you get

average_precision_score(y_true, y_scores, average=None)
# array([0.58333333, 0.33333333])

whose second entry agrees with the answer for the positive class, and whose average gives you the other output you're seeing.

---

As for hand calculations:
For the positive class, with a threshold in $(0.8,1]$ we get zero positive predictions, and so 0 recall and 1 precision (by convention). With the threshold in $(0.4,0.8)$ we get one false positive, one false negative, and one true negative, so 0 recall and 0 precision. With the threshold in $[0, 0.4)$ we get one true positive and two false positives, so 1 recall and 0.333 precision. So the table from the linked question in this case is

    R     P
1   0.0   1.0   
2   0.0   0.0   
3   1.0   0.333   

Finally, the average precision computation is $$(0.0-0.0)\cdot 0.0 + (1.0-0.0)\cdot 0.333 = 0.333$$ Somewhat degenerate example, but it checks out.

For the other class: $(0.6,1]$ gives zero "positive" predictions, so 0 and 1 again. $(0.2,0.4)$ gives one true positive, one false positive, and one true negative. $[0,0.2)$ gives two true positives, one false positive. So

    R     P
1   0.0   1.0   
2   0.5   0.5   
3   1.0   0.666   

and average precision is $$(0.5-0.0)\cdot 0.5 + (1.0-0.5)\cdot0.666 = 0.58333$$