Naive Bayes is called naive because it makes the naive assumption that features have zero correlation with each other. They are independent of each other. Why does naive Bayes want to make such an assumption?
$\begingroup$
$\endgroup$
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
2
By doing so, the joint distribution can be found easily by just multiplying the probability of each feature whilst in the real world they may not be independent and you have to find the correct joint distribution. It is naive due to this simplification.
answered Sep 2, 2018 at 11:40
-
1$\begingroup$ Thanks for answer. Upvoted and selected as answer. Given this naive assumption, is feature engineering particularly important for naive bayes in selecting the right features which are truly independent of each other? Is fewer features better to minimize the possibility for error? $\endgroup$ Commented Sep 2, 2018 at 11:54
-
1$\begingroup$ Generally, naive Bayes is a very simple and is not used most of the time. There are other learning approaches that are better for learning mappings. About the second part, it highly depends on your task. $\endgroup$ Commented Sep 2, 2018 at 12:17
$\begingroup$
$\endgroup$
Naive bayes make such assumptions to simplify the calculations. You can take a look at bayesian belief network which do not make such assumptions
answered Sep 4, 2018 at 8:44
$\begingroup$
$\endgroup$
Just to complete the answers given and clarify them in some points: the assumption in Naïve Bayes is that features are conditionally independent given the predicted variable, not independent. Note also that, even though this simplification makes naïve assumptions about the conditional joint distribution of features that are in many cases far from the true distribution, our aim here is not to estimate probabilities but to perform a binary classification and, for that purpose, our simplification strategy may be good enough.
