1
$\begingroup$

To classify my samples, I decided to use Naive Bayes classifier, but I coded it, not used built-in library functions.

If I use this equality, I obtain nice classification accuracy: p1(x) > p2(x) => x belongs to C1

However, I could not understand why discriminant functions produce negative values. If they are probability functions, I think they must generate a value between 0 and 1.

Is there anyone who can explain the reason ?

$\endgroup$
1
$\begingroup$

In Naive Bayes, for the case of two classes, a discriminant function could be $$D(\boldsymbol{x}) = \frac{P(\boldsymbol{x}, c=1)}{P(\boldsymbol{x}, c=0)}$$ which can be anywhere in $[0, +\infty)$, and decides $c=1$ if $D(\boldsymbol{x})>1$, $c=0$ otherwise, or it could be the logarithm of that value

$$d(\boldsymbol{x}) = \text{log}\frac{P(\boldsymbol{x}, c=1)}{P(\boldsymbol{x}, c=0)}=\text{log}P(\boldsymbol{x}, c=1)-\text{log}P(\boldsymbol{x}, c=0)$$ which can be anywhere in $(-\infty, +\infty)$ (handling zero probability as a special case), and decides $c=1$ if $d(\boldsymbol{x})>0$, $c=0$ otherwise.

As a side note, $P(\boldsymbol{x}, c=k)$ in Naive Bayes is calculated as $$P(\boldsymbol{x}, c=k)=P(c=k)\prod_{i=1}^{d}P(x_i|c=k)$$ or equivalently for log probabilities as $$\text{log}P(\boldsymbol{x}, c=k)=\text{log}P(c=k) + \sum_{i=1}^{d}\text{log}P(x_i|c=k)$$

$\endgroup$
  • 1
    $\begingroup$ Thank you so much. While I was designing my bayes classifier, I took natural logarithm of posterior probability to make probability formula less complicated. Because I benefited from normal distribution function when I define probability discriminant function. $\endgroup$ – Goktug Mar 28 at 19:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.