1
$\begingroup$

I have been reading the book "The Data Science Design Manual" (by Steven S. Skiena) and I came across an example that explained how the Bayes theorem can be applied that confused me and made me suspect it might be wrong. The example is the following:

$$ P(A|B) = \frac{P(B|A)P(A)}{P(B)} $$ Suppose A is the event that person x is actually a terrorist, and B is the result of a feature-based classifier that decides if x looks like a terrorist. When trained/evaluated on a data set of 1,000 people, half of whom were terrorists, the classifier achieved an enviable accuracy of, say, 90%. The classifier now says that Skiena looks like a terrorist. What is the probability that Skiena really is a terrorist? The key insight here is that the prior probability of “x is a terrorist” is really, really low. If there are a hundred terrorists operating in the United States, then P(A) = 100/300,000,000 = 3.33 × 10−7 . The probability of the terrorist detector saying yes, P(B) = 0.5, while the probability of the detector being right when it says yes P(B|A) = 0.9. Multiplying this out gives a still very tiny probability that I am a bad guy, $$ P(A|B) = \frac{P(B|A)P(A)}{P(B)} = \frac{(0.9)(3.33x10^{-7})}{0.5} = 6x10^{-7} $$

However, $P(B) = 0.5$ doesn't seem correct to me. $P(B)$ is supposed to be the probability of the terrorist detector saying yes when exercised on a person randomly selected from the United State's population (e.g. Skiena). If I understand correctly, this $0.5$ used by the author is the percentage of terrorists in the evaluation data set for the classifier, which is not the same thing for several reasons:

  • This is a sample that is not randomly selected to be equivalent to some population (the one Skiena is selected from), but specifically selected to contain the aforementioned ratio of terrorists.
  • This ratio is not the ratio of people in the evaluation dataset that look like terrorists (i.e. the probability the classifier would say yes for a random person in that sample), but the ratio of actual terrorists in the sample.

My understanding is that in order to calculate $P(B)$ more properly one would have to draw a random sample from the United States population (assuming this is where Skiena is picked up from), then run the classifier on them and calculate the percentage of people the classifier said yes for.

Is my thinking correct or am I missing something?

$\endgroup$

1 Answer 1

1
$\begingroup$

P(B) - the output of the model for an observation - is not going to change in this case. The model is deterministic with respect to the inputs. Skiena can be scored multiple times and will get the same score.

You are questioning if P(B)=0.5 is actually a 50% chance. That is a good question. It is called calibration. Is the output of the model well-calibrated. Many (most) of the models I have dealt with are not well calibrated. They are simply rank-ordering. A 0.5 is higher than a 0.4 therefore the 0.5 is closer to the event (Y=1, riskier, more similar, etc.). There are some algorithms that build models closer to well-calibrated and some post-processing that can be done for calibration. Your solution is about determining if the model is well-calibrated and if not, how to adjust the output to be well-calibrated.

It is a great question - What does the score actually mean? Is it rank-order only or is it also well-calibrated.

In this example, it seems the author is implying the model is well-calibrated. Hence the P(B) notation instead of "the output of B" or some similar notation. Occam's razor. So reading the question, assume P(B) = 50% chance.

Different metrics go for calibration vs rank ordering. Many people like AUROC which is for rank ordering. AUROC gives no insight into calibration.

$\endgroup$
5
  • $\begingroup$ I read about calibration of classifiers, but I am still struggling to understand how the author came up with the number 0.5 for P(B), assuming it corresponds to the probability the detector say yes when asked whether a (randomly selected) person looks like a terrorist. $\endgroup$
    – Dimos
    May 9, 2022 at 16:39
  • $\begingroup$ I do not know. Often in problem in books or classes these things are given. Like P(A) = 3.33 × 10−7. That came from somewhere. Is that true or guess or made up? It will not be true since we do not know there are 100 terrorists here - or we would do something about it. Is it a guess from reliable sources or just a number? Some data in problems is there to do the problem. $\endgroup$
    – Craig
    May 11, 2022 at 8:55
  • $\begingroup$ no, the excerpt I shared is all the data provided for that problem. So, I assumed the number 0.5 is inferred from that data. You can see the full description using google books if you search for the first sentence (e.g. shorturl.at/oprAV), if you want (it's section 7.13). $\endgroup$
    – Dimos
    May 11, 2022 at 11:44
  • $\begingroup$ Then I think you need to ask the author where the number came from. If it is a real number, they can cite their sources and/or methods. $\endgroup$
    – Craig
    May 12, 2022 at 9:29
  • $\begingroup$ FWIW, the author's answer was: "It says that the training set half of the examples were terrorists, so the classifier wants to say yes or no with equal frequency" I don't think that really elucidates things for me, since a finely tuned classifier would not have a 50% probability to detect a terrorist on the full population (it wouldn't act based on the proportion of terrorists in the training set, but the feature values of a person). $\endgroup$
    – Dimos
    Aug 13, 2022 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.