# How does ,the Mutlinomial Bayes's alpha parameter, affects the text classification task?

I would like to know how the alpha parameter, in Multinomial Bayes, affects the text classification task.

I know that this parameter is correlated to the algorithm's ability in classifying unseen words during training. How changes text classification using low or high values of alpha?

• Are you referring to the alpha parameter in the laplace smoothing ? – moksha Apr 18 '18 at 11:16
• @moksha Yes, exactly. – Simone Apr 18 '18 at 11:25

$$P(I\,have\,huge\,respect\,for\,women) = P(I)\times P(have)\times P(huge)\times P(respect)\times P(for)\times P(women)\\$$ $$\\$$ $$P(I\,have\,huge\,respect\,for\,women|Trump) = P(I|Trump)\times P(have|Trump)\times P(huge|Trump)\times P(respect|Trump)\times P(for|Trump)\times P(women|Trump)\\$$ $$\\$$ $$P(I\,have\,huge\,respect\,for\,women|Not\,Trump) = P(I|Not\,Trump)\times P(have|Not\,Trump)\times P(huge|Not\,Trump)\times P(respect|Not\,Trump)\times P(for|Not\,Trump)\times P(women|Not\,Trump)$$ $$\\$$ Depending on the probablity of equations (2) and (3) the user would make a decision whether the statement was made by Trump or not. Now lets say that none of the 5 training set tweets have the word huge in it, in which case, $$\\$$ $$P(huge|Trump)=P(huge|Not\,Trump)=0$$ $$\\$$ And hence, equations (2) and (3) are now zero and this is bad. A solution to this is to do smoothing or rather Laplace smoothing. The basic idea is to increase the probablities of all bigrams in your non-maximum likelihood equation (Since we are changing the counts from what they occurred in hopes to make it better) by 1 to make everything non-zero. That increase by 1 is called adding the pseudocount or as you know it as $\alpha$. Now, $\alpha=1$ may not always give the most accurate probablities and hence they could attain any finite non-negative integer. The way to know what $\alpha$ gives the most accurate responses is through iterating over all values of $\alpha$ on the training set unfortunately. The way to do that is whole different topic.