Given that I have a word that does not occur in any of my documents:
newword, and given that I have two classes:
class2. For instance, the total number of words in
class1 is 3, the total number of words in
class2 is 6, and the number of unique words in all documents is 8.
Also, given the Naive Bayes formula, the word
newword will have a higher probability of belonging to
class1 (because the denominator will be lower while the numerator stays the same). Is there any statistical/logical theoretical explanation (that drives the algorithm) about this behavior?
In a nutshell, I just want to know what is the motivation of giving higher probability to classes with fewer words.
Ps.: I am using Laplace Smoothing. Therefore:
P(newword|class1) = 0 + 1 / 3 + 8 = 0.09 >
P(newword|class2) = 0 + 1 / 6 + 8 = 0.07