Naive about which Naive Bayes in article

Question

Working with the Naive Bayes spam filtering article on Wikipedia (https://en.wikipedia.org/wiki/Naive_Bayes_spam_filtering)

Is it a binary multinomial equation or a Bernoulli form or some other?

In this 2006 paper, it discusses that there are many Naive Bayes algorithms: Spam Filtering with Naive Bayes – Which Naive Bayes? http://www.aueb.gr/users/ion/docs/ceas2006_paper.pdf

The paper states that binary Multinomial NB performs best.

If it is not multinomial, what changes are needed to make it so?

Answer 1

First off, yes there are different Naive Bayes algorithms. But they are all based on the same principle, namely Bayes Theorem where the features are assumed to be independent.

Here is a short guide on when to use which for spam detection (or document classification in general for that matter):

• Bernoulli Naive Bayes

  Each e-mail is represented as a binary vector, and in this case, whether a word is present in the e-mail matters as much as if it isn't. For instance, the fact that "viagra" is in the e-mail might mean it's "spam" and the fact it isn't might mean it's "not spam"

• Multinomial Naive Bayes with boolean features

  Each e-mail is represented as a binary vector, and in this case, whether a word is present in the e-mail matters more than if it isn't. For instance, if "grandma" is in the e-mail, it might mean that it's "not spam", but if it isn't in the e-mail, it doesn't necessarily mean that it's "spam".

• Multinomial Naive Bayes with term frequency

  Each e-mail is represented by how many times each word occurs.

• Gaussian Naive Bayes

  This deals with continuous values, so this isn't applicable here.

Answer 2

Both screengrabs you've posted are not associated with any particular method of calculating probabilities. So they keep the formulae for calculating Naive Bayes very general, without endowing the notation with any specific meaning.

To make them "anything", whether some variant of Bernoulli, Multinomial or so on, involves defining how the $P(...)$s or $p$s are defined and evaluated.

Since you want to use this paper, you can simply "plug in" formulae to evaluate them as defined in the paper.

For example, for multivariate Bernoulli Naive Bayes with a Laplacian prior, you would plug in:

Similarly, for Multinomial Naive Bayes (whether using TF or binary attributes), you would plug in:

As for the second formula, $p = \frac{1}{1+e^{\eta}}$, it is simply a more computationally-accurate way to evaluate the general formula Naive Bayes posterior calculation $p = \frac{\prod_{i=1}^N p_i}{\prod_{i=1}^N p_i + \prod_{i=1}^N (1-p_i)}$. This is because performing summation operations in the log space over many operands may often be more accurate than performing multiplication operations in the usual space, due to the way floating-point arithmetic is implemented in computers.