Since NB is a generative classifier, we assume that the data points are all generated from a distribution, right?

But since we can compute the MLE of p(x_i|y) by counting (MLE of p(x_i|y) = # of (x_i,y) / # of y), do we really need a specific distribution, if at all, to model the likelihood(p(x_i|y))?

  • $\begingroup$ What about a test document that has no similar training points? Clearly you need a representative training set. $\endgroup$ – paparazzo Nov 6 '15 at 20:51
  • $\begingroup$ I don't quite follow the question. Naive Bayes is simply a concept of breaking the conditional probability due to independence. It's distribution independent. You can do it with our MLE if you wish. $\endgroup$ – SmallChess Nov 7 '15 at 2:54
  • $\begingroup$ Yes it's distribution independent. But we may need to assume a distribution to perform MLE. The distribution can be any arbitrary distribution, though. $\endgroup$ – AsymTrick Nov 7 '15 at 22:24

The most harrowing problem would be "interpolation". That is, computing $p(x_i | y)$ for $x_i$ and $y$ you have not yet observed, but fall within the convex hull of your observations.

You have to bin events together, or come up with a wacky interpolation scheme. This does work. However, If you assume a distribution, $p(x_i | y)$ becomes continuous, and you end up with a more statisically sound means of interpolation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.