1
$\begingroup$

Does Naive Bayes classifier require a loss function for Bernoulli classification? If yes, what loss function does Naive Bayes classification use? And how does it work?

$\endgroup$
2

1 Answer 1

2
$\begingroup$

You can check this source as a nice explanation of Naive Bayes and applications. The way Naive Bayes classifier is based on Bayes theorem:

enter image description here

where the objective function in this algorithm is to maximize the posterior probability (assuming features are independent from each other, something which is not realistic but still works on many applications):

enter image description here

The simple way of working of Naive Bayes is actually calculating the conditional probability and the priors, which is no other than by counting the times a feature value appears in smaples of class j, the total samples of such class, etc:

enter image description here

The discrimination of when a sample belongs to one class or another is in fact decided by the highest posterior probability, but the loss function is actually a zero-one decision, and it is robust to the violation of the attributes independence assumption (check this paper). This loss function just assigns a cost of 1 for misclassifications, and 0 for right classifications, so does not take into account any margin in between as other loss functions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.