Why people use precision and not true negative rate (specificity)?

Question

From my experience the standard way to evaluate a classifier is not to check only its accuracy but also its recall and precision.

People tend to assume that recall measures the model on positive samples, and precision measures it on negative samples. In addition, those measures are considered as a good way to evaluate a model when you have a non-ballanced distribution.

However, precision does not measure the model performance only on negative samples. In addition, in contrast to recall, it is not agnostic to positive-negative distribution. In fact, when recall is a fixed attribute of the model, precision is an attribute of the model and the positive-negative distribution.

Recall is the True Positive Rate, and there is a symmetric measure - True Negative Rate, with all of its good qualities.

So my question is - Why, in the ML field, people tend to use recall and precision and not recall and true negative rate?

Answer 1

The positive/negative distinction is not what the precision/recall pair of measures tries to capture.

• precision measures the proportion of correctly predicted instances among the instances predicted as positive. In other words, if X is the precision then one can say "when the classifier predicts an instance as positive, it is correct X% of the time".
• recall measures the proportion of correctly predicted instances among the gold-standard positive instances. In other words, if Y is the recall then one can say "when an instance is gold-positive, the classifier identifies it correctly Y% of the time".

Thus the pair precision/recall focuses on the distinction "predicted as true" vs. "gold-standard true". It is designed for tasks where the positive class is really the class of interest, and the negative class is not essential.

Of course it's also a matter of habit: people get used to thinking in terms of precision/recall, but technically other measures (like TPR vs. TNR) could also be appropriate.

---

[Update]

Is it desirable for the evaluation measure to be agnostic to the class distribution?

I don't know if there is a canonical answer to this question, I suspect that there isn't and that it's a matter of interpretation. Personally I don't have a strong answer to this question, but at least it doesn't seem obvious to me that it is.

Let's look at an example where the difference appears:

Confusion matrix A

Gold positive = 10%

	pred pos	pred neg
gold pos	2	8
gold neg	10	80

• Precision=0.16, Recall=0.2
• TPR=0.2, TNR=0.89

Confusion matrix B

Gold positive = 50%

	pred pos	pred neg
gold pos	2	8
gold neg	1	8

• Precision=0.66, Recall=0.2
• TPR=0.2, TNR=0.89

Interpretation

• First it's important to keep in mind that when the class distribution is different, it means that the problem itself is different. Since in general evaluation measures are used to compare systems on the same task, there's no strong reason to favour an evaluation which is constant across different class distributions.
• In the example above the precision is higher in case B, where TNR is equal in the two cases.
  • In the case of precision, the higher performance in case B cannot be interpreted as "B is better than A", since the problem is different (as mentioned above; btw this is a mistake that we see quite often on DSSE, when resampling is wrongly done on both the training and test set). This would be an argument leaning in favour of TNR.
  • However it could be interpreted as an indication that the problem in case B is easier than in case A. It's indeed easier in general to find the positive instances when they represent a larger proportion of the data. The fact that TNR cannot give this kind of information may sometimes be a disadvantage.
• The precision/recall pair is intended for problems where the focus is the positive class, which is usually the minority class. A typical example is the detection of a disease in a population. Intuitively, in this kind of problem knowing the precision score is more useful than knowing the TNR. Also the TNR is usually very high and its variations are small.