Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
To classify my samples, I decided to use Naive Bayes classifier, but I coded it, not used built-in library functions.
If I use this equality, I obtain nice classification accuracy: p1(x) > p2(x) => x belongs to C1
However, I could not understand why discriminant functions produce negative values. If they are probability functions, I think they must generate a value between 0 and 1.
Is there anyone who can explain the reason ?
In Naive Bayes, for the case of two classes, a discriminant function could be $$D(\boldsymbol{x}) = \frac{P(\boldsymbol{x}, c=1)}{P(\boldsymbol{x}, c=0)}$$ which can be anywhere in $[0, +\infty)$, and decides $c=1$ if $D(\boldsymbol{x})>1$, $c=0$ otherwise, or it could be the logarithm of that value
$$d(\boldsymbol{x}) = \text{log}\frac{P(\boldsymbol{x}, c=1)}{P(\boldsymbol{x}, c=0)}=\text{log}P(\boldsymbol{x}, c=1)-\text{log}P(\boldsymbol{x}, c=0)$$ which can be anywhere in $(-\infty, +\infty)$ (handling zero probability as a special case), and decides $c=1$ if $d(\boldsymbol{x})>0$, $c=0$ otherwise.
As a side note, $P(\boldsymbol{x}, c=k)$ in Naive Bayes is calculated as $$P(\boldsymbol{x}, c=k)=P(c=k)\prod_{i=1}^{d}P(x_i|c=k)$$ or equivalently for log probabilities as $$\text{log}P(\boldsymbol{x}, c=k)=\text{log}P(c=k) + \sum_{i=1}^{d}\text{log}P(x_i|c=k)$$
