Rationale behind most published works in medical imaging trying to reduce false positives

Question

In medical image processing, most of the published works try to reduce false positive rate (FPR) while in reality, false negatives are more dangerous than false positives. What is the rationale behind it?

Answer 1

TL;DR: diseases are rare, so the absolute number of false positives is a lot more than that of false negatives.

Let's assume that our system has the same false positive and false negative rate of 1% (pretty good!), and that we're detecting the presence of new cancers this year: 439.2 / 100,000 people, or 0.5% of the population. [source]

• No cancer, no detection: 99.5% x 99% = 98.5% (98.505%)
• No cancer, detection: 99.5% x 1% = 1.0% (0.995%)
• Cancer, detection: 0.5% x 99% = 0.5% (0.495%)
• Cancer, no detection: 0.5% x 1% = 0.005%

So we can see that we have a problem: for everyone who has cancer, two people who didn't have cancer wind up with invasive surgery, chemotherapy or radiotherapy.

For every person who fails to have a present cancer detected, two hundred people receive actively harmful treatment they didn't need and can't really afford.

Answer 2

You know the story of the boy who cried wolf, right?

It's the same idea. After some classifier gives false alarms (cries wolf) so many times, the medical staff will turn it off or ignore it.

"Oh, this again! NOPE!"

At least with the bioengineering group I've worked with, the emphasis is on reducing FPR specifically because the goal is to make a tool that will alert physicians to potential pathology, and they've told us that they will ignore a product that cries wolf too much.

For a product that aids physicians, we have to appeal to their psychology, despite the legitimate argument that missing the wolf on the farm is worse than crying wolf.

Edit: Decreasing false positives also has a legitimate argument. If your computer keeps crying wolf while getting the occasional true positive (and catching most of the true positives), it's effectively saying that someone might be sick. They're in the hospital. The physician knows that the patient might be sick.

Answer 3

Summary: the question probably* isn't whether one false negative is worse than one false positive, it's probably* more like whether 500 false positives are acceptable to get down to one false negative.

* depends on the application

---

Let me expand a bit on @Dragon's answer:

• Screening means that we're looking for disease among a seemingly healthy population. As @Dragon explained, for these we need extremely low FPR (or high Sensitivity), otherwise we'll end up with many more false positives than true positives. I.e., the Positive Predictive Value (# truly diseased among all diagnosed positive) would be inacceptably low.

• Sensitivity (TPR) and Specificity (TNR) are easy to measure for a diagnostic system: take a number of truly (non)diseased cases and measure the fraction of correctly detected ones.

• OTOH, both from doctors' and patients' point of view, the predicitive values are more to the point. They are the "inverse" to Sensitivity and specificity and tell you among all positive (negative) predictions, what fraction is correct. In other words, after the test said "disease" what is the probability that the patient actually does have the disease.

• As @Dragon showed you, the incidence (or prevalence, depending on what test we're talking about) plays a crucial role here. Incidence is low in all kinds of screening/early cancer diagnosis applications.
  To illustrate this, ovarian cancer screening for post-menopausal women has a prevalence of 0.04 % in the general population and 0.5 % in high-risk women with family history and/or known mutations of tumor suppressor genes BRCA1 and 2 [Buchen, L. Cancer: Missing the mark. Nature, 2011, 471, 428-432]

• So the question is typically not whether one false negative is worse than one false positive, but even 99 % specificity (1 % FPR) and 95 % sensitivity (numbers taken from the paper linked above) then means roughly 500 false positives for each false negative.

• As a side note, also keep in mind that early cancer diagnosis in itself is no magic cure for cancer. E.g. for breast cancer screening mammography, only 3 - 13 % of the true positive patients actually benefit from the screening.
  So we also need to keep an eye on the number of false positives for each benefitting patient. E.g. for mammography, together with these numbers, a rough guesstimate it that we have somewhere in the range of 400 - 1800 false positives per benefitting true positive (39 - 49 year old group).

• With hundreds of false positives per false negative (and also maybe hundreds or even thousands of false positives per patient benefitting from the screening) the situation isn't as clear as "is one missed cancer worse than one false positive cancer diagnosis": false positives do have an impact, ranging from psychological and psycho-somatic (worrying that you have cancer in itself isn't healthy) to physical risks of follow-up diagnoses such as biopsy (which is a small surgery, and as such comes with its own risks).
  Even if the impact of one false positive is small, the corresponding risks may add up substantially if hundreds of false positives have to be considered.

  Suggested reading: Gerd Gigerenzer: Risk Savvy: How to Make Good Decisions (2014).

• Still, what PPV and NPV are needed to make a diagnostic test useful is highly dependend on the application.
  As explained, in screening for early cancer detection the focus is usually on PPV, i.e. making sure you do not cause too much harm by false negatives: finding a sizeable fraction (even if not all) of the early cancer patients is already an improvement over the status quo without screening.
  OTOH, HIV test in blood donations focuses first on NPV (i.e. making sure the blood is HIV-free). Still, in a 2nd (and 3rd) step, false positives are then reduced by applying further tests before worrying people with (false) positive HIV test results.

• Last but not least, there are also medical testing applications where the incidences or prevalences aren't as extreme as they usually are in screening of not-particularly-high-risk populations, e.g. some differential diagnoses.

Answer 4

From a personal perspective, rather than a data science experience, a false positive has a higher impact on the patient's quality of live than a false negative (at least in most applications of medical image processing. We're not talking about lab results here).

Let's look at a concrete example: tumor screening.

A false negative means that an early-stage tumor has more time to grow and develop into malicious cancer. Overall this process takes a long time and each subsequent screening has a higher chance to detect it, but realistically the long-term health of a patient suffers.

Additionally, there's always a human involved in diagnosing. Medical image processing at it's current technological stage is meant to be a help for medical personell, not a substitute. It's often meant to point out lesions or changes in tissue that are so subtle that a human might overlook them. There's no chance a doctor would overlook an advanced stage tumor. They don't need image processing for that.

In terms of medical procedures, if a tumor doesn't become inoperable before the next screening, there's no big difference between removing an early-stage tumor or one that had a little more time to grow. The amount of tissue removed is more, but the kind of operation is often the same. (This assumes that the patient does regular health screenings.)

A false positive has many implications that are not all directly related to an ailment:

• Additional procedures. After an imaging process yields a positive result, more tests are conducted for which blood or tissue is extracted (biopsy). Objectively speaking the body of the patient is damaged to be able to verify the imaging result.
• Fear. Lab tests take time. The person affected often lives through several days, sometimes weeks, of uncertainty weather or not the lesion is actually cancer. Many people who have experienced such a false positive describe this event as "traumatizing" and suffer from health-related anxiety for a long time.
• Time investment. If verifying the imaging result via lab tests or similar takes several examinations, the patient and doctors have to invest time for them. Even if it takes only one test, there are several people involved, including nurses, doctors and lab technicians. In a time when doctors are chronically overworked, this should be avoided if possible.
• Unnecessary medication. In the worst case the patient is treated for an ailment they don't even have and their body is put under unnecessary strain by side effects of medication.
• Loss of effect. Medical personell will ignore true positive results if a procedure yields too many false positives (as explained in other answers).

This risk-benefit-evaluation shows that a false negative includes less risk for a patient than a false positive. Therefore the priority of reducing false positives is generally higher.

Answer 5

Clinician's time is precious

From within the field of medicine, clinicians often have a wide variety of illnesses to try to detect and diagnose, and this is a time consuming process. A tool that presents a false positive (even if at a low rate) is less useful because it's not possible to trust that diagnosis, meaning every time it makes that diagnosis, it needs to be checked. Think of it like the WebMD of software - everything is a sign of cancer!

A tool that presents false negatives, but always presents true positives, is far more useful, as a clinician doesn't need to waste time double-checking or second guessing the diagnosis. If it marks someone as being ill with a specific diagnosis, job done. If it doesn't, the people which aren't highlighted as being ill will receive additional tests anyway.

It's better to have a tool that can accurately identify even a single trait of an illness, than a tool that maybe fudges multiple traits.

Answer 6

False Positive Rate (FPR) also known as false alarm rate (FAR); A large False Positive Rate can produce a poor performance of the Medical Image Detection System. A false positive is where you receive a positive result for a test, when you should have received a negative results. For example A pregnancy test is positive, when in fact the person isn't pregnant.

Answer 7

In all likelihood, everyone on this thread already knows that this is a problem at the core of Bayesian analysis. Solely for the benefit of future pilgrims who might think of false positives as somehow solely a problem in radiology, I hope this comment will provide a little more general perspective.

Answer 8

Several studies demonstrated [1, 2] that radiologist fatigue levels and performance are related to environmental factors such as number of False-Positives and False-Negatives. However, the number of False-Positives is usually higher than the number of False-Negatives if we consider solely the diagnosis (e.g., breast, lung cancers, etc.) over medical imaging analysis. This is due to the fact that typically, clinicians do not want to take risks [2] in the presence of an ambiguous case. Thus, clinicians will diagnose that case with their higher probability for the worse case scenario. In breast cancer diagnosis, such medical decision will translate into a patient's biopsy in which patients are physically and mentally affected, as well as affected by healthcare costs. On the other hand, False-Negatives are severe medical errors that can lead to patient mortality.

Although False-Negatives are more severe, the improvement window is much lower. For instance, in breast cancer diagnosis, about 8% to 10% of cases are yielding False-Negatives and between 30% to 50% are yielding False-Positives [3]. These results show a much more promising output relation for working on False-Positives than False-Negatives. Moreover, it is much safer for the Data Science community to work on False-Positives improvement for now, as it instantiates lower levels of patient mortality.

References

[1] Stephen Waite, Srinivas Kolla, Jean Jeudy, Alan Legasto, Stephen L. Macknik, Susana Martinez-Conde, Elizabeth A. Krupinski, Deborah L. Reede, Tired in the Reading Room: The Influence of Fatigue in Radiology, Journal of the American College of Radiology, Volume 14, Issue 2, 2017, Pages 191-197, ISSN 1546-1440, DOI: https://doi.org/10.1016/j.jacr.2016.10.009

[2] Francisco Maria Calisto, Carlos Santiago, Nuno Nunes, Jacinto C. Nascimento, Introduction of human-centric AI assistant to aid radiologists for multimodal breast image classification, International Journal of Human-Computer Studies, Volume 150, 2021, 102607, ISSN 1071-5819, DOI: https://doi.org/10.1016/j.ijhcs.2021.102607

[3] Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018 Nov;68(6):394-424. doi: 10.3322/caac.21492. Epub 2018 Sep 12. Erratum in: CA Cancer J Clin. 2020 Jul;70(4):313. PMID: 30207593.