2
$\begingroup$

I am currently studying the online material of Stanford CS229 and I came across the likelihood function for discriminative(for example, regression) and generative algorithms(for example, naive bayes): +Discriminative:

Discriminative likelihood function

+Generative:

Generative likelihood function

In both cases, m is the number of training examples and all training examples are independent of each other. What I am wondering is why in discriminative likelihood function, the formula is the product of conditional probability of y given x and in generative likelihood function, the formula is the product of joint probability of x and y? Is there some reasoning behind this choice?

Improve this question
$\endgroup$
3
  • $\begingroup$ It might be a simple mistake, but "i.e." implies equality. Discriminative models are not synonymous with regression, and generative models are not synonymous with Naive Bayes. $\endgroup$ – Emre Jun 8 '17 at 15:57
  • $\begingroup$ Oh yeah I am sorry for the mistake as english is not my first language. I'm just trying to show an example for each discriminative and generative model. $\endgroup$ – CuriousAlpaca Jun 8 '17 at 18:14
  • 1
    $\begingroup$ That's okay, I assumed you meant "e.g.". To answer your question, generative models allow you to generate samples from the joint distribution. Discriminative models don't learn the distribution of X so reducing complexity if it is not desired (don't model what you don't need). Generative models, on the other hand, allow you to marginalize over missing data. $\endgroup$ – Emre Jun 8 '17 at 18:31
2
$\begingroup$

Discriminative model learn to classify an x into class y: the conditional probability distribution p(y|x).

Generative models learn the joint probability distribution p(x,y).It can be transformed into p(y|x) by a Bayes rule.

You get different likelihood functions because the way those models classify data is different.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.