1
$\begingroup$

What happens if we flip the arrows in a Naive Bayes classifier? To clarify - from what I have found naive Bayes is defined for the following network structure:

enter image description here

I'm interested to understand what happens if instead from y->x the child will be y. As shown in the following:

enter image description here

I'm not entirely sure if we get the same results as a regular naive Bayes classifier (intuitively we don't), and if not how to estimate both the likelihood of the network and the parameters?

For further clarification: lets say there is a training set of size n; each data point consisting of (x,y) where x is a vector of size m of binary values and y is the class and is also binary. E.g. (0,1,0,1,1) where the last index is the class. I'm trying to figure out the likelihood of the network (where y is the child) and how to estimate the parameters in the case of y being the child.

$\endgroup$
  • $\begingroup$ This depends on how you interpret this. If you are looking at these pictures as a Directed Acyclic Graph (DAG) such as in a Bayesian Network, then the top picture depicts $X_1, X_2, X_3 | Y$ while the bottom shows $Y | X_1, X_2, X_3$. However, if it's not a DAG, then both are equivalent. $\endgroup$ – Jon Dec 11 '17 at 22:14
1
$\begingroup$

To elaborate on my comment, the first image depicts conditional dependence of $X_i | Y$ for $i \in 1,...,m$. Therefore, your likelihood for this model would be

$$ \begin{aligned} P(Y = y) \times P(X_1 | Y = y) \times ... \times P(X_m | Y = y) \end{aligned} $$

The second image (bottom) depicts $Y | X_i$, therefore the likelihood would be

$$ P(X_1) \times P(X_2) \times ... \times P(Y | X_1, X_2, ..., X_m) \\ \ = P(X_1) \times P(X_2) \times ... \times P(Y | X_1) \times ... \times P(Y | X_m) $$

Assuming that both scenarios do not have conditional independence, the two are not equivalent!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.