2
$\begingroup$

I came across an earlier post that was resolved and had a follow up to it but I couldn't comment because my reputation is under 50. Essentially I am interested in calculating the denominator in Naive Bayes. enter image description here

Now I understand that the features in Naive Bayes are assumed to be independent so could we calculate $p(x) = p(x_{1})p(x_{2})...p(x_{n})$ or would we have to use this formula $$p(\mathbf{x}) = \sum_k p(C_k) \ p(\mathbf{x} \mid C_k)$$ with the conditional independence assumption that$$ p(\mathbf{x} \mid C_k) = \Pi_{i} \, p(x_i \mid C_k) $$

My question is would both ways of calculating give the same p(x)?

Link to the original question : https://datascience.stackexchange.com/posts/69699/edi

Edit** : Sorry I believe the features have conditional independence, rather than complete independence. Therefore it is incorrect to use $p(x) = p(x_{1})p(x_{2})...p(x_{n})$?

Lastly, I understand we don't actually need the denominator to find our probabilities but am asking out of curiosity.

$\endgroup$
1
1
$\begingroup$

The way to calculate $p(x)$ is indeed:

$$p(x) = \sum_k p(C_k) \ p(x| C_k)$$

Since in general one needs to calculate $p(C_k,x)$ (numerator) for every $k$, it's simple enough to sum all these $k$ values. It would be incorrect to use the product, indeed.

Lastly, I understand we don't actually need the denominator to find our probabilities but am asking out of curiosity.

Calculating the marginal $p(x)$ is not needed in order to find the most likely class $C_k$ because:

$$argmax_k(\{ p(C_k|x) \}) = argmax_k(\{ p(C_k,x) \})$$

However it's actually needed to find the posterior probability $p(C_k | x)$, that's why it's often useful to calculate the denominator $p(x)$ in order to obtain $p(C_k | x)$, especially if one wants to output the actual probabilities.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.