Does Naive Bayes classifier require a loss function for Bernoulli classification? If yes, what loss function does Naive Bayes classification use? And how does it work?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ mm I think it is the zero-one loss function as the generic loss for Naive Bayes $\endgroup$– German C MDec 16, 2021 at 14:40
-
$\begingroup$ The loss function of naive Bayes is always the negative joint log-likelihood, -log p(X, Y) $\endgroup$– DeshwalDec 16, 2021 at 16:00
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You can check this source as a nice explanation of Naive Bayes and applications. The way Naive Bayes classifier is based on Bayes theorem:
where the objective function in this algorithm is to maximize the posterior probability (assuming features are independent from each other, something which is not realistic but still works on many applications):
The simple way of working of Naive Bayes is actually calculating the conditional probability and the priors, which is no other than by counting the times a feature value appears in smaples of class j, the total samples of such class, etc:
The discrimination of when a sample belongs to one class or another is in fact decided by the highest posterior probability, but the loss function is actually a zero-one decision, and it is robust to the violation of the attributes independence assumption (check this paper). This loss function just assigns a cost of 1 for misclassifications, and 0 for right classifications, so does not take into account any margin in between as other loss functions.
