2
$\begingroup$

There is a example in sklearn.metrics.average_precision_score documentation.

import numpy as np
from sklearn.metrics import average_precision_score
y_true = np.array([0, 0, 1, 1])
y_scores = np.array([0.1, 0.4, 0.35, 0.8])
average_precision_score(y_true, y_scores)  
0.83

But when I plot precision_recall_curve

precision, recall, _ = precision_recall_curve(y_true, y_scores)
plt.plot( recall,precision)

I got the picture:enter image description here

why the area under the precision_recall_curve is not 0.83?

$\endgroup$
4
$\begingroup$

According to the documentation, the value is not exactly the area under curve, it is $$\text{AP} = \sum_n(R_n - R_{n-1})P_n.$$ which is a rectangular approximation.

For your specific example, i.e.

      R     P
  1   0.0   1.0   
  2   0.5   1.0   
  3   0.5   0.5   
  4   0.1   0.66

it is calculated as $$\begin{align*} \text{AP} & = \overbrace{(0.5 - 0.0)\times1.0}^{(R_2 - R_1)P_2} + \overbrace{(0.5 - 0.5)\times 0.5}^{(R_3 - R_2)P_3} + \overbrace{(1.0 - 0.5) \times0.66}^{(R_4 - R_3)P_4} \\ &= 0.5 + 0.00+ 0.33 = 0.83 \end{align*}$$ which is the area under the red curve as illustrated below:

compared to $$\text{AUPR}=0.5 + \overbrace{\frac{0.5 + 0.66}{2} \times 0.5}^{\text{trapezoid area}} = 0.79$$ which is the area under the blue curve.

|improve this answer|||||
$\endgroup$
  • 3
    $\begingroup$ Beat me to it; in particular, the documentation states "This implementation is not interpolated and is different from computing the area under the precision-recall curve with the trapezoidal rule, which uses linear interpolation and can be too optimistic." $\endgroup$ – Ben Reiniger May 17 '19 at 20:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.