Since NB is a generative classifier, we assume that the data points are all generated from a distribution, right?

But since we can compute the MLE of p(x_i|y) by counting (MLE of p(x_i|y) = # of (x_i,y) / # of y), do we really need a specific distribution, if at all, to model the likelihood(p(x_i|y))?

  • $\begingroup$ What about a test document that has no similar training points? Clearly you need a representative training set. $\endgroup$
    – paparazzo
    Nov 6, 2015 at 20:51
  • $\begingroup$ I don't quite follow the question. Naive Bayes is simply a concept of breaking the conditional probability due to independence. It's distribution independent. You can do it with our MLE if you wish. $\endgroup$
    – SmallChess
    Nov 7, 2015 at 2:54
  • $\begingroup$ Yes it's distribution independent. But we may need to assume a distribution to perform MLE. The distribution can be any arbitrary distribution, though. $\endgroup$
    – AsymTrick
    Nov 7, 2015 at 22:24

1 Answer 1


The most harrowing problem would be "interpolation". That is, computing $p(x_i | y)$ for $x_i$ and $y$ you have not yet observed, but fall within the convex hull of your observations.

You have to bin events together, or come up with a wacky interpolation scheme. This does work. However, If you assume a distribution, $p(x_i | y)$ becomes continuous, and you end up with a more statisically sound means of interpolation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.