1
$\begingroup$

My understanding of generative models is that they generate data to match certain statistical properties. Intuitively, I find it hard how generative models can be used for classification purposes. On the other hand, discriminative models being used for classification is self-explanatory.

$\endgroup$

1 Answer 1

1
$\begingroup$

You will find an explanation on Wikipedia. But let us sum up:

Given your Data $D$ you are interested in target values $Y$, your classifications. Discriminative models are, like your said, the straight forward way of modelling the correspondence of your target given your data, $P(Y|D)$. The generative model, on the other hand, calculates $P(D|Y)$.

Let us consider the example of e-mail spam filtering. We have a set of reference e-mails and a label for each mail, which indicates whether it is spam. If we now, for example, look at Naive Bayes, we can see it utilizes the Bayes Formula to calculate the posterior estimate $P(Y|X) = \frac{P(X|Y)P(X)}{P(Y)}$. Opposed to other Bayesian Inference methods, Naive Bayes does not hold $P(X)$ as a model, but $P(X|Y)$, which can be modelled by our reference e-mail set. In this domain, all e-mails can be modelled as equally distributed, thus making $P(X)$ uninteresting. $P(Y)$ can also be approximated by the reference set (only looking at the labels).

This way, we can generate probabilities for our label $Y$, which is binary in our case. Now we can apply the maximum a posteriori approach: We calculate $P(Y=true|X)$ and $P(Y=false|X)$ and see which of both is more likely. The most likely option will be selected as our classification.

$\endgroup$
2
  • 1
    $\begingroup$ The word "likely" is important here, and may be the root of the OP's skepticism. This intuitive skepticism is healthy, as "likely" and "always" are very different, and understanding outcome error of the models is an important thing. $\endgroup$
    – davmor
    Commented Sep 3, 2018 at 22:34
  • $\begingroup$ Surely it is very important to see that, but it is not restricted to generative models. Discriminative models can fail in the same way - it's good practice to be skeptic, whatever model you are using. $\endgroup$
    – André
    Commented Sep 4, 2018 at 7:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.