Imbalanced data: understanding example from Bishop PRML book?

Question

I'm trying to understand the 3-step procedure to compensate for the effects of imbalanced data described in Section 1.5.4 - pg 45 of Bishop's PRML book. Please refer to the following excerpt from the book.

"Compensating for class priors. Consider our medical X-ray problem again, and suppose that we have collected a large number of X-ray images from the general population for use as training data in order to build an automated screening system. Because cancer is rare amongst the general population, we might find that, say, only 1 in every 1,000 examples corresponds to the presence of cancer. If we used such a data set to train an adaptive model, we could run into severe difficulties due to the small proportion of the cancer class. For instance, a classifier that assigned every point to the normal class would already achieve 99.9% accuracy and it would be difficult to avoid this trivial solution. Also, even a large data set will contain very few examples of X-ray images corresponding to cancer, and so the learning algorithm will not be exposed to a broad range of examples of such images and hence is not likely to generalize well. A balanced data set in which we have selected equal numbers of examples from each of the classes would allow us to find a more accurate model. However, we then have to compensate for the effects of our modifications to the training data. Suppose we have used such a modified data set and found models for the posterior probabilities. From Bayes’ theorem (1.82), we see that the posterior probabilities are proportional to the prior probabilities, which we can interpret as the fractions of points in each class. We can therefore simply take the posterior probabilities obtained from our artificially balanced data set and first divide by the class fractions in that data set and then multiply by the class fractions in the population to which we wish to apply the model. Finally, we need to normalize to ensure that the new posterior probabilities sum to one. Note that this procedure cannot be applied if we have learned a discriminant function directly instead of determining posterior probabilities. "

Some notations:

$C_{1}$ : class that indicates the presence of cancer.

$p(C_{1}|X)$ and $p(C_{1})$: posterior and prior probabilities of the imbalanced data

$\tilde{p}({C}_{1}|X)$ and $\tilde{p}({C}_{1})$: posterior and prior probabilities of the balanced data

Question:

Following the procedure described in the above excerpt from the book, is it correct to write that $$ p(C_{1}|X)=\frac{\tilde{p}({C}_{1}|X)p(C_{1})}{\tilde{p}({C}_{1})} \quad \mbox{?} $$

If so, what would the proof be?

Answer 1

Bayes rule for the balanced conditions should be

$\tilde{p}(C_1|X) = \frac{\tilde{p}(C_1|X)\tilde{p}(C_1)}{\tilde{p}(X)}$

and that is what is given in the book (rather than the denominator being $\tilde{p}(C_1)$, as given in the question).

Likewise, Bayes rule for the imbalanced data is

$p(C_1|X) = \frac{p(C_1|X)p(C_1)}{p(X)}$

However, to make things easier, we will ignore the denominators for the moment and look at unnormalised probabilities:

$\tilde{p}(C_1|X) \propto \tilde{q}(C_1|X) = \tilde{p}(C_1|X)\tilde{p}(C_1)$

and

$p(C_1|X) \propto q(C_1|X) = p(C_1|X)p(C_1)$

Next, note that $\tilde{p}(C_1|X) = p(C_1|X)$ because it just describes how the data belonging to class $C_1$ are distributed, and that does not depend on the prior probabilities, so

$p(C_1|X) \propto q(C_1|X) = \tilde{p}(C_1|X)p(C_1) = \tilde{q}(C_1|X)\frac{p(C_1)}{\tilde{p}(C_1)}$

So we can obtain the un-normalised probability of class membership for the imbalanced conditions from the corresponding un-normalised probability for balanced conditions by multiplying by the ratio of class prior probabilities.

All we need to do then is to re-normalise so that the probabilities add up to one again, for a two class problem:

$p(C_1|X) = \frac{q(C_1|X)}{q(C_1|X) + q(C_2|X)} = \frac{\tilde{p}(X|C_1)p(C_1)}{\tilde{p}(X|C_1)p(C_1) + \tilde{p}(X|C_2)p(C_2)} = \frac{\tilde{p}(X|C_1)p(C_1)}{\tilde{p}(X)}.$

and that is the desired result.