24
$\begingroup$

Can anyone give me some examples where precision is important and some examples where recall is important ?

$\endgroup$
  • $\begingroup$ f1-score is the way to go, my friend $\endgroup$ – Neoares Apr 27 '18 at 9:58
  • $\begingroup$ More than what's more important between the two you should ask what are the cases where you want to maximise one over the other (which doesn't necessarily make the other "less" important). $\endgroup$ – gented Sep 21 '19 at 15:18
35
$\begingroup$
  • For rare cancer data modeling, anything that doesn't account for false-negatives is a crime. Recall is a better measure than precision.
  • For YouTube recommendations, false-negatives is less of a concern. Precision is better here.
| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ @fateh The major difference is FP vs FN. YouTube recommendation don't place emphasis on FN but hospital clinical decisions must. $\endgroup$ – HelloWorld Apr 26 '18 at 14:46
12
$\begingroup$

I can give you my real case when recall is more important:

We have thousands of free customers registering in our website every week. The call center team wants to call them all, but it is imposible, so they ask me to select those with good chances to be a buyer (with high temperature is how we refer to them). We don't care to call a guy that is not going to buy (so precision is not important) but for us is very important that all of them with high temperature are always in my selection, so they don't go without buying. That means that my model needs to have a high recall, no matter if the precision goes to hell.

I hope it helps! Miguel.

| improve this answer | |
$\endgroup$
7
$\begingroup$

Although in some situations recall may be more important than precision (or vice versa), you need both to get a more interpretable assessment.

For instance, as noted by @SmallChess, in the medical community, a false negative is usually more disastrous than a false positive for preliminary diagnoses. Therefore, one might consider recall to be a more important measurement. However, you could have 100% recall yet have a useless model: if your model always outputs a positive prediction, it would have 100% recall but be completely uninformative.

This is why we look at multiple metrics:

| improve this answer | |
$\endgroup$
7
$\begingroup$

Which is more important simply depends on what the costs of each error is.

Precision tends to involve direct costs; the more false positives you have, the more cost per true positive you have. If your costs are low, then precision doesn't matter as much. For instance, if you have 1M email addresses, and it will cost $10 to send an email to all of them, it's probably not worth your time to try to identify the people most likely to respond, rather just spamming all of them.

Recall, on the other had, tends to involve opportunity costs; you are giving up opportunities every time you have a false negative. So recall is least important when the marginal value of additional correct identification is small, e.g. there are multiple opportunities, there is little different between them, and only a limited number can be pursued. For instance, suppose you want to buy an apple. There are 100 apples at the store, and 10 of them are bad. If you have a method of distinguishing bad apples that misses 80% of good ones, then you will identify about 18 good apples. Normally, a recall of 20% would be terrible, but if you only want 5 apples, then missing those other 72 apples doesn't really matter.

So recall is most important when:

-The number of opportunities is small (if there were only 10 good apples, then you would be unlikely to find 5 good ones with a recall rate of only 20%)
-There are significant differences between opportunities (if some apples are better than others, then a recall rate of 20% is enough to get 5 good apples, but they aren't necessarily going to be the best apples)
OR
-The marginal benefit of opportunities remains high, even for a large number of opportunities. For instance, while most shoppers won't have much benefit from more than 18 good apples, the store would like to have more than 18 apples to sell.

Thus, precision will be more important than recall when the cost of acting is high, but the cost of not acting is low. Note that this is the costs of acting/not acting per candidate, not "cost of having any action at all" versus "cost of not having any action at all". In the apple example, it's the cost of buying/not buying a particular apple, not the cost of buying some apples versus the cost of not buying any apples; the cost of not buying a particular apple is low because there are lots of other apples. Since the cost of buying a bad apple is high, but the cost of passing up a particular good apple is low, precision is more important in that example. Another examples would be hiring when there's a lot of similar candidates.

Recall is more important than precision when the cost of acting is low, but the opportunity cost of passing up on a candidate is high. There's the spam example I gave earlier (the cost of missing out on an email address isn't high, but the cost of sending out an email to someone who doesn't respond is even lower), and another example would be identifying candidates for the flu shot: give the flu shot to someone who doesn't need it, and it costs a few dollars, don't give it to someone who does need it, and they could die. Because of this, health care plans will generally offer the flu shot to everyone, disregarding precision entirely.

| improve this answer | |
$\endgroup$
3
$\begingroup$

Accumulation has a great answer on how you can come up with more examples explaining the importance of precision over recall and vice versa.

Most of the other answers make a compelling case for the importance of recall so I thought I'd give an example on the importance of precision. This is a completely hypothetical example but it makes the case.

Let us say that a machine learning model is created to predict whether a certain day is a good day to launch satellites or not based on the weather.

  • If the model accidentally predicts that a good day to launch satellites is bad (false negative), we miss the chance to launch. This is not such a big deal.

  • However, if the model predicts that it is a good day, but it is actually a bad day to launch the satellites(false positive) then the satellites may be destroyed and the cost of damages will be in the billions.

This is a case where precision is more important than recall.

| improve this answer | |
$\endgroup$
2
$\begingroup$

I had a tough time remembering the difference between precision and recall, until I came up with this mnemonic for myself:

PREcision is to PREgnancy tests as reCALL is to CALL center.

With a pregnancy test, the test manufacturer needs to be sure that a positive result means the woman is really pregnant. People might react to a positive test by suddenly getting married or buying a house (if many consumers got false positives and suffered huge costs for no reason, the test manufacturer would lack customers). I got a false negative pregnancy test once, and it just meant it took a few more weeks before I found out I was pregnant...the truth ultimately became apPARENT. (Pun intended.)

Now picture a call center for insurance claims. Most fraudulent claims are phoned in on Mondays, after the fraudsters connect with collaborators and craft their made-up stories ("let's say the car was stolen") over the weekend. What's the best thing for an insurance company to do on Mondays? Maybe they should tune to favor recall over precision. It is far better to flag more claims as positive (likely fraud) for further investigation than to miss some of the fraud and pay out cash that should have never been paid. A false positive (flagged for additional scrutiny as possibly fraud, but the customer loss was real) can likely be cleared up by assigning an experienced adjustor, who can insist on a police report, request building security video, etc. A false negative (accepting a fraudster's false claim and paying out in cash) is pure loss to the insurance company, and encourages more fraud.

F1 is great but understanding how the test/prediction will be used is really important, because there's always some risk of being wrong...you want to know how dire the consequences will be if wrong.

| improve this answer | |
$\endgroup$
1
$\begingroup$

Email Spam detection:This is one of the example where Precision is more important than Recall.

Quick Recap:

  • Precision: This tells when you predict something positive, how many times they were actually positive. whereas,

  • Recall: This tells out of actual positive data, how many times you predicted correctly.

Having said above, in case of spam email detection, One should be okay if a spam email (positive case) left undetected and doesn't go to spam folder but, if an email is good (negative), then it must not go to spam folder. i.e. Precison is more important. (If model predicts something positive (i.e. spam), it better be spam. else, you may miss important emails).

Hope it clarifies.

| improve this answer | |
$\endgroup$
0
$\begingroup$

When we have imbalanced class and we need high true positives, precision is prefered over recall. because precision has no false negative in its formula, which can impact.

| improve this answer | |
$\endgroup$
0
$\begingroup$

Here's a simple example that I took from Aurelion Geron's book, Hands-on Machine Learning with Scikit-Learn and Tensorflow. Imagine that we want to make sure that our web site blocker for our child only allows 'safe' websites to be shown.

In this case, a 'safe' website is the positive class. Here, we want the blocker to be absolutely certain that the website is safe, even if some safe websites are predicted to be part of the negative or unsafe class and are consequently blocked. That is, we want high precision at the expense of recall.

In the case of airport security, where a safety risk is the positive class, we want to make sure that every potential safety risk is investigated. In this case, we will have high recall at the expense of precision (a lot of bags where there are no safety hazards will be investigated).

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.