14
$\begingroup$

Naive Bayes apparently handles missing data differently, depending on whether they exist in training or testing/classification instances.

When classifying instances, the attribute with the missing value is simply not included in the probability calculation (reference)

In training, the instance [with the missing data] is not included in frequency count for attribute value-class combination. (reference)

Does that mean that particular training record simply isn't included in the training phase? Or does it mean something else?

$\endgroup$
2
  • 1
    $\begingroup$ Maybe Laplace Smoothing is what you are looking for? en.wikipedia.org/wiki/Additive_smoothing $\endgroup$ Dec 16, 2014 at 18:59
  • 1
    $\begingroup$ about the comment, please note that 'missing value' is different to 'zero probability'. Zero probability means that we know the value and it is zero. But missing value means that we don't know the probability. It 'could be' zero. But it could also be 0.75 or 0.3 or any value between 0 and 1. But we just don't know that. And we use laplacian smoothing to handle zero probability problem. not missing value problem. for missing value problem, we just ignore that attribute like given in the above answer. $\endgroup$ Oct 14, 2017 at 14:22

1 Answer 1

18
$\begingroup$

In general, you have a choice when handling missing values hen training a naive Bayes classifier. You can choose to either

  1. Omit records with any missing values,
  2. Omit only the missing attributes.

I'll use the example linked to above to demonstrate these two approaches. Suppose we add one more training record to that example.

Outlook  Temperature  Humidity   Windy   Play
-------  -----------  --------   -----   ----
rainy    cool        normal    TRUE    no
rainy    mild        high      TRUE    no
sunny    hot         high      FALSE   no
sunny    hot         high      TRUE    no
sunny    mild        high      FALSE   no
overcast cool        normal    TRUE    yes
overcast hot         high      FALSE   yes
overcast hot         normal    FALSE   yes
overcast mild        high      TRUE    yes
rainy    cool        normal    FALSE   yes
rainy    mild        high      FALSE   yes
rainy    mild        normal    FALSE   yes
sunny    cool        normal    FALSE   yes
sunny    mild        normal    TRUE    yes
NA       hot         normal    FALSE   yes
  1. If we decide to omit the last record due to the missing outlook value, we would have the exact same trained model as discussed in the link.

  2. We could also choose to use all of the information available from this record. We could choose to simply omit the attribute outlook from this record. This would yield the following updated table.

           Outlook            Temperature           Humidity   
====================   =================   =================  
          Yes    No            Yes   No            Yes    No 
Sunny       2     3     Hot     3     2    High      3     4
Overcast    4     0     Mild    4     2    Normal    7     1 
Rainy       3     2     Cool    3     1
          -----------         ---------            ---------- 
Sunny     2/9   3/5     Hot   3/10   2/5    High    3/10   4/5 
Overcast  4/9   0/5     Mild  4/10   2/5    Normal  7/10   1/5
Rainy     3/9   2/5     Cool  3/10   1/5


            Windy        Play
=================    ========
      Yes     No     Yes   No
False 7      2       10     5
True  3      3
      ----------   ----------
False  7/10    2/5   10/15  5/15
True   3/10    3/5

Notice there are 15 observations for each attribute except Outlook, which has only 14. This is since that value was unavailable for the last record. All further development would continue as discussed in the linked article.

For example in the R package e1071 naiveBayes implementation has the option na.action which can be set to na.omit or na.pass.

$\endgroup$
1
  • $\begingroup$ +1 for organized tables, helped me debug my own code as well $\endgroup$ Sep 25, 2020 at 1:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.