Are Naive Bayes algorithms affected by outliers in the data? Suppose there is a data set, does one need to remove outliers before applying Naive Bayes?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
There are different flavors of Naive Bayes, so the answer depends a bit on the use case.
One potential issue with outliers is that unseen observations can lead to 0 probabilities. For example, Bernoulli Naive Bayes applied to word features will always produce 0 probabilities when it encounters a word that wasn't seen in the training data. Outliers in this sense can be a problem. However, all these and similar issues of Naive Bayes have well-known solutions (like Laplace smoothing, i.e. adding an artificial count for every word) and are routinely implemented.
In Gaussian Naive Bayes, outliers will affect the shape of the Gaussian distribution and have the usual effects on the mean etc.
So depending on your use case, it still makes sense to remove outliers.
$\begingroup$
$\endgroup$
Yes outlier affect naive bayes. If a word that comes in testing data that has not been seen in training leads to zero probab of that particular word in the particular class. And we know in naive bayes we multipy probab of words lying in that particular class and results zero..that leads to wrong result so thats why we have laplace smoothing in naive bayes.. You can also remlve the high and low idf words from the text to remove outlier which doesnt contribute to classification.
