11
$\begingroup$

Today, in a lecture it was claimed that the direction of edges in a Bayes network doesn't really matter. They don't have to represent causality.

It is obvious that you cannot switch any single edge in a Bayes network. For example, let $G = (V, E)$ with $V = \{v_1, v_2, v_3\}$ and $E=\{(v_1, v_2), (v_1, v_3), (v_2, v_3)\}$. If you would switch $(v_1, v_3)$ to $(v_3, v_1)$, then $G$ would no longer be acyclical and hence not a Bayes network. This seems to be mainly a practical problem how to estimate the probabilities then. This case seems to be much more difficult to answer, so I will skip it.

This made me ask the following questions for which I hope to get answers here:

  1. Is it possible for any directed acyclical graph (DAG) to reverse all edges and still have a DAG?
  2. Assume a DAG $G$ and data is given. Now we construct the inverse DAG $G_\text{inv}$. For both DAGs, we fit the data to the corresponding Bayes networks. Now we have a set of data for which we want to use the Bayes network to predict the missing attributes. Could there be different results for both DAGs? (Bonus if you come up with an example)
  3. Similar to 2, but simpler: Assume a DAG $G$ and data is given. You may create a new graph $G'$ by inverting any set of edges, as long as $G'$ remains acyclical. Are the Bayes networks equivalent when it comes to their predictions?
  4. Do we get something if we have edges which do represent causality?
Improve this question
$\endgroup$
1
  • $\begingroup$ I'm doubtful, if the accepted answer is correct. AFAIK, representational power is same irrespective of direction of the edges in a BN. But, using cause-->effect direction typically requires lesser edges in the graph (DAG). $\endgroup$
    – KGhatak
    Apr 2 at 8:44

3 Answers 3

Reset to default
8
+50
$\begingroup$

TL;DR: sometimes you can make an equivalent Bayesian network by reversing arrows, and sometimes you can't.

Simply reversing the direction of the arrows yields another directed graph, but that graph is not necessarily the graph of an equivalent Bayesian network, because the dependence relations represented by the reversed-arrow graph can be different from those represented by the original graph. If the reversed-arrow graph represents different dependence relations than the original, in some cases it's possible to create an equivalent Bayesian network by adding some more arrows to capture dependence relations which are missing in the reversed-arrow graph. But in some cases there is not an exactly equivalent Bayesian network. If you have to add some arrows in order to capture dependencies, you might end up with a graph which represents fewer independence relations and therefore fewer opportunities for simplifying the computations of posterior probabilities.

For example, a -> b -> c represents the same dependencies and independencies as a <- b <- c, and the same as a <- b -> c, but not the same as a -> b <- c. This last graph says that a and c are independent if b is not observed, but a <- b -> c says a and c are dependent in that case. We can add an edge directly from a to c to capture that, but then a and c being independent when b is observed is not represented. That means that there is at least one factorization which we cannot exploit when computing posterior probabilities.

All this stuff about dependence/independence, arrows and their reversals, etc., is covered in standard texts on Bayesian networks. I can dig out some references if you want.

Bayesian networks don't express causality. Judea Pearl, who did a lot of work on Bayesian networks, has also worked on what he calls causal networks (essentially Bayesian networks annotated with causal relations).

Improve this answer
$\endgroup$
4
  • $\begingroup$ This answers questions (2) and (3). Do you have an idea about question (1) and (4), too? (Yes, references would be nice) $\endgroup$
    – Martin Thoma
    Apr 20, 2016 at 3:45
  • $\begingroup$ (1) Consider the contrapositive: if the reversed-arrow graph has a directed cycle, then following the arrows around the cycle backwards must be a directed cycle in the original graph. (4) Bayesian networks are probabilistic models and as such don't represent causality. It's possible that some arrows actually refer to causal relations, but this is lost in a probabilistic model. Maybe a causes b, but a -> b and a <- b are equally valid probabilistic models. $\endgroup$
    – Robert Dodier
    Apr 20, 2016 at 16:02
  • 1
    $\begingroup$ Some introductory references. Koller & Friedman: "Probabilistic Graphical Models". Cowell, Dawid, Lauritzen, and Spiegelhalter: "Probabilistic Networks and Expert Systems". Castillo, Gutierrez, and Hadi: "Expert Systems and Probabilistic Network Models". $\endgroup$
    – Robert Dodier
    Apr 20, 2016 at 16:11
  • $\begingroup$ You can reverse any arrow and get an equivalent Bayesian Network as long as you keep the v-structures (do not reverse an arrow which another arrow is also pointing to the node it is pointing at) $\endgroup$
    – borgr
    Jun 30, 2018 at 13:08
1
$\begingroup$

This might be a bit unsatisfying, so feel free not to accept this answer, and apologies in advance.

In a Bayes net, nodes represent random variables, and edges represent conditional dependences. When you interpret the nodes a certain way, conditioning flows a certain way naturally. Arbitrarily reversing them doesn't really make sense in the context of modeling data. And a lot of time, the arrows do represent causality.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ This is pretty far off the mark. The "natural" interpretation is something imposed on the model, it's not part of the model itself. You can reverse dependencies (adding additional edges as necessary to preserve the set of dependencies represented by the network) and it's still a Bayesian network. Whether it makes sense isn't answerable by examining just the network itself. Incidentally Judea Pearl, one of the big movers behind Bayesian networks in the 80's and 90's, has been working more recently on formal models for causality, which do express causal relationships in the model. $\endgroup$
    – Robert Dodier
    Apr 19, 2016 at 16:29
  • $\begingroup$ You say, "whether it makes sense isn't answerable by examining just the network itself." I never said it was. I said "when you interpret the nodes a certain way, conditioning flows a certain way..." This probably reflects my bias; you can call the stuff I work on a bayes net, but this question would never come about for me. For example, if two nodes represent the same variable at different times, there would be no question about which direction the conditioning flows. I accept the possibility, however, that there are situations where people might use these Baye's nets in a less rigid way. $\endgroup$
    – Taylor
    Apr 21, 2016 at 0:15
0
$\begingroup$

Question 3

synergy.st-andrews.ac.uk/vannesmithlab claims that the graphs

G1 = o->o->o and
G2 = o<-o->o

are in one equivalence class. According to that source, the models represents exactly the same joint probability distribution.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ That can't be right. For G1, the first and last are dependent in the absence of any known values. For G2, the first and last are not dependent in the absence of any known values. Did you mean to write G2 = o <- o -> o instead? Anyway I'm not seeing a claim about those particular graphs on the web page which you've referenced; perhaps you can be more specific. $\endgroup$
    – Robert Dodier
    Apr 19, 2016 at 22:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.