i looked into one of the post about naive bayes calulation of naive part

Predit the class label for instance (A=1,B=2,C=2) using naive Bayes classifcation.

Let C1 be class 1 and C2 be class 2.

enter image description here

For C1, by the assumption of Naive Bayesian Classifier, we have P(A=1,B=2,C=2∣C1)=P(A=1∣C1)⋅P(B=2∣C1)⋅P(C=2∣C1)

Take P(A=1∣C1) as an example. There are 4 training records of C1, among which there are 2 records with A=1. Therefore, P(A=1∣C1)=24. Similarly, you can calculate P(B=2∣C1) and P(C=2∣C1).

It is similar to calculate P(A=1,B=2,C=2∣C2).

My query is how can we calculate the NON Naive part of this ?

P(A=1,B=2,C=2∣C1)= P(A=1∣C1)⋅P(B=2∣C1)⋅P(C=2∣C1) -> Here the events are considered to be independent of each other

What if the events are dependent ? How should we calcualte in that case ?


1 Answer 1


I think in practice you would make the assumption of independence in any case. Considering all the dependencies can get quite complicated very quickly and you'd need many observations. For example, in your example A=1, B=2 and C=2 was never observed,but you can't really tell, if that is because P(A=1 , B=2, C=2 | C1) is small and there is too little data, or if P(A=1 , B=2, C=2 | C1)=0. In many practical applications the approximation with independence introduces less error than not having enough observations for all possible events.

There is hybrid approaches, though, where you would model some dependence, e.g. tree-augmented naive bayes. These approaches usually perform better, but are computationally more complicated as well.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.