4
$\begingroup$

Let's have a Naive Bayes Bernoulli classifier with $n_C$ classes and $n_F$ features.

According to the formula in here and here and almost every theory book I could see, Laplacian smoothing means that we take

$$P(x=X|c=C)=\frac{\#\{x=X,c=C\} +\alpha}{\#\{c=C\} +n_F\alpha }$$

but in the implementations of sklearn and this book the formula is actually

$$P(x=X|c=C)=\frac{\#\{x=X,c=C\} +\alpha}{\#\{c=C\} +2\alpha}$$

why is the number of features taken as two? Is there a theoretical reason?

$\endgroup$
1
$\begingroup$

I think the answer is just that in the case of the Bernoulli model, your variables are binary variables which means the only values they can take are 0 or 1. Therefore the "number of features" also called "size of the vocabulary" in the case of text classification is limited to 2.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.