Avoiding the zero problem

Question

I have a dataset and I'm trying to predict the label for my sample but I couldn't map it since that case never showed here is my sample (I'm using naïve Bayesian method)

X=(Age=middle, has_job=false, own_house=true, credit_rating=good)

and that's it the dataset

What I'm supposed to do to fix the problem? I know that I should avoid it but didn't know how

Answer 1

I totally agree with Esmailian.

Naive Bayes is Naive - Assumes Independence.

Steps:

• Calculate Independently
• Smooth using Laplacian smoothing (to avoid zeroing the whole value)

Additional Tip:

• Use log instead of multiplying the probabilities. (this will make sure that your values are not closing to zero, keeping some context.)

Example:

$$p(a) = p(x1)\cdot p(x2)$$

Applying logarithm,

$$\log(p(a)) = \log(p(x_1)) + \log(p(x_2))$$

As you need to finally classify, taking the log won't hurt.

Answer 2

Naive Bayes considers each feature separately, i.e. features are independent given the class. The exact X is not in the training data, but each of its features has been seen before.

However, there is still a problem with P(own_house=true|No) which is zero according to training data (0 divided by 6). For this, we use Laplace smoothing to replace the zero with (0+1)/(6+4)=1/10. Now, Naive Bayes could assign X to a class.

Naive Bayes classifier compares

P(X, Class=Yes) = P(Class=Yes) * P(Age=middle|Yes) * P(has_job=false|Yes) * P(own_house=true|Yes) * P(credit_rating=good|Yes) = 9/15 * 3/9 * 4/9 * 6/9 * 4/9 = 0.0263

with

P(X, Class=No) = P(Class=No) * P(Age=middle|No) * P(has_job=false|No) * P(own_house=true|No) * P(credit_rating=good|No) = 6/15 * 2/6 * 6/6 * 1/10 * 2/6 = 0.0044

and assigns X to Class = Yes.