Why there is no alpha argument ( smoothing parameter in Laplace smoothing) for GaussianNB() in sklearn library? ? Although BernoulliNB() and MultinomialNB() have an alpha parameter but GaussianNB() doesn't have ?
1 Answer
$\begingroup$
$\endgroup$
2
Because smoothing makes sense only for BernoulliNB and MultinomialNB, which have categorical features, whereas GaussianNB works with numerical features which are supposed to follow a normal distribution.
For BernoulliNB and MultinomialNB, the features often represent frequencies and can be zero. In theory, this can cause one or several of the conditional probabilities to be zero, and therefore occasionally make the posterior probability zero. It can even cause the posterior to be zero for all the classes, an inconsistency. Smoothing prevents these issues.
In GaussianNB the conditional probability is calculated from the normal distribution so they can never be zero, thus this problem cannot happen.
-
$\begingroup$ What confuses me is that the probability that a continuous random variable equals some value is always zero , like here stats.stackexchange.com/questions/60702/… $\endgroup$– AAACommented Oct 11, 2022 at 22:58
-
$\begingroup$ @AAA As far as I remember this is usually handled by taking the probability of the interval around the value instead, i.e. instead of calculating $p(x)$ one calculates $p(x \in [x-\epsilon/2, x+\epsilon/2])$ with $\epsilon$ a fixed constant, e.g. 0.000001. This would be specific to GaussianNB of course. $\endgroup$– ErwanCommented Oct 12, 2022 at 9:33