Naive Bayes classifier works on the principal of conditional independence. Take an example, a bank manager wants to know how risky it is to approve loan for a customer depending upon customers background. Like the income,job,account balance, previous loan history, property of customer etc. Now we often see that to find out the risky or not risky for a loan approval naive Bayes classifier works well for prediction.However the attributes that I defined in the above they are definitely depended on each other. For example if a customer has job he or she has income, if the customer has income then he or she has good account balance, also good credit history. So, for the real world example the attributes are depended on each other.

Then how could Naive Bayes classifier gives good result? Is not it violets it's own principal which is conditional independence.

Thank you.


1 Answer 1


The standard proof just shows that Naive Bayes works when there is conditional independence. It does not say that it doesn't work when this condition is violated.

Wikipedia cites this article (H.Zhang, The Optimality of Naive Bayes). The article basically says that the method still keeps working even when there are dependencies as long as their total contribution is "not too large".


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.