I've been searching around for an explanation to this, and haven't come across one yet- in scikit-learn, when I compute the auc() of the precision_recall_curve(), I get a different macro-metric than when I use the builtin average_precision_score() function. As per the docs, this makes sense, due to auc() using a trapezoidal approximation, and thus average precision being more accurate. However, when I compute the micro-metrics, the values are the same. I'm trying to understand this.
Here's an example:
yhat_raw = np.array([[0.511, 0.62, 0.3],[0.11, 0.76, 0.945],[0.002, 0.671, 0.452],[0.9102, 0.783, 0.2]])
y = np.array([[0, 1, 1],[0, 0, 1],[0, 0, 0],[0, 1, 1]])
yhatmic = yhat_raw.ravel()
ymic = y.ravel()
prec["micro"], rec["micro"], _ = precision_recall_curve(ymic, yhatmic,
pos_label=1)
auc(rec["micro"], prec["micro"])
returns 0.58305555555555544.
average_precision_score(ymic, yhatmic) returns 0.62222222222222223.
Upon actually deploying the model, these metrics are coming to the same thing. I think this makes sense, considering that at a larger scale (e.g. along a dimension of thousands of datapoints rather than len(rec['micro']) = 11 datapoints in this example), the trapezoidal approximation is much closer to the true value.
My question is, why do the metrics then differ in the macro case? Is it fair to say that in a real-world deployment, the micro-evaluation has enough datapoints for the trapezoidal approximation to converge to the true value, but the macro- does not (computed across ~3000 test examples, with a label space of ~2000 labels)?
