# Problem understanding probabilistic generative models for classification

I am a student and I am studying machine learning. I am focusing on probabilistic generative models for classification and I am having some troubles understanding this topic.

In the slide of my professor it is written the following:

which I don't understand.

So far, I have understood that in the generative probailistic models, we ant to estimate $$P(C_i|x)$$, which is the probability of having class $$i$$ given a data $$x$$, using the likelihood and the Bayes theorem.

So, it starts by writing the Bayes rule, but the the slides says that we can write this as a sigmoid, but why?

If I have to try to give an answer to it, I would say because the sigmoid gives a number from $$0$$ to $$1$$, and so a probability, but it is just a guess I am doing.

Moreover, it continues by saying that we can use a gaussian distribution for $$P(x|C_i)$$, and so $$P(x|C_i)=N(\mu ,\sigma )$$, and so :

I don't know if my question is clear so sorry if it is not but I am really confused. If it is not lcear please tell me I will try to edit it. Thanks in advance.

Note: if it can be useful, this has been taken from the Bishop book at page 197

• What book is the Bishop book? Aug 31, 2020 at 18:51

My understanding is:

1. The first line is OK - it derives from Bayes Rule

2. Assume that this probability follows a logistic function, that is that

$$P(C_1|x) = \frac{1}{1+exp(-a)}$$

1. Then if

$$y = \frac{1}{1+exp(-z)}$$

then

$$z = ln(\frac{y}{1-y})$$

(some lines below but with $$a$$ and $$\sigma$$)

1. Therefore:

$$a = ln(\frac{P(C_1|x)}{1-P(C_1|x)})$$

1. If there are only two classes then $$1-P(C_1|x)= P(C_2|x)$$.

2. Then

$$a = ln(\frac{P(C_1|x)}{P(C_2|x)}$$

1. Using Bayes on $$P(C_1|x)$$ and $$P(C_2|x)$$

$$a = ln(\frac{P(x|C_1) P(C_1)}{P(x|C_2)P(C_2)}$$

1. The whole thing now is to model $$P(x|C_i)$$. He assumes that it is a normal distribution. And from there you should the second set of equations. The important part (as far as I remember) is that the two covariance matrix (for $$C_1$$ and for $$C_2$$) are the same. I do not have access to the book now but I will look it up tomorrow and see if there is any difficult step in the derivations.

tldr: The whole point is he assumes that the probability as a logistic, then he gets the formula for the $$a$$, then he assumes a normal distribution for $$P(x|C_i)$$, then he assumes same covariance matrix.