1
$\begingroup$

Learning about different Machine Learning concepts, I came across Generative and Discriminative model. To infer from what I have studied, generative model is based on P(x,y)(Joint probability distribution) whereas discriminative model is based on P(y|x) which is conditional probability. But the question is why; infact HOW? How does joint probability give rise to new data meanwhile conditional probability just works on current dataset? Can anyone explain me with an intuitive example or maybe some link which would elucidate the concept ?

$\endgroup$

1 Answer 1

2
$\begingroup$

The difference between discriminative and generative models is a common question which has some very good answers on Cross Validated, in particular here and here.

But the question is why; infact HOW? How does joint probability give rise to new data meanwhile conditional probability just works on current dataset?

The word "generative" in generative models doesn't mean that the model generates actual new data additionally to the dataset. It refers to the nature of the theoretical model, in the sense that the generative approach assumes that any sample of data is generated from some distribution, and it tries to estimate this distribution. Once the distribution is estimated the model could be used to actually generate instances following this distribution, but:

  • usually that's not the goal, the goal typically being to predict the probability of a new instance using the model (inference)
  • normally this data should not be reused for estimation (i.e. training), since it is artificial data. Additionally it would be pointless since the best that can be achieved is to re-estimate the same model.
  • naturally the model obtained from estimation is only as good as the assumptions made for its design, and there are usually many such assumptions.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.