10
$\begingroup$

This question asks about generative vs. discriminative algorithm, but can someone give an example of the difference between these forms when applied to Natural Language Processing? How are generative and discriminative models used in NLP?

Improve this question
$\endgroup$
1
  • $\begingroup$ @knb The link to the video is broken $\endgroup$
    – viebel
    Apr 29, 2018 at 15:58

1 Answer 1

Reset to default
8
$\begingroup$

Let's say you are predicting the topic of a document given its words.

A generative model describes how likely each topic is, and how likely words are given the topic. This is how it says documents are actually "generated" by the world -- a topic arises according to some distribution, words arise because of the topic, you have a document. Classifying documents of words W into topic T is a matter of maximizing the joint likelihood: P(T,W) = P(W|T)P(T)

A discriminative model operates by only describing how likely a topic is given the words. It says nothing about how likely the words or topic are by themselves. The task is to model P(T|W) directly and find the T that maximizes this. These approaches do not care about P(T) or P(W) directly.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.