Which tribe does Probabilistic Graphical Models fall under?

Question

Pedro Domingos in "The Master Algorithm" listed five tribes of machine learning algorithms:

• Symbolists
• Connectionists
• Evolutionaries
• Bayesians
• Analogizers

Which category do probabilistic graphical models fall under?

From wikipedia (https://en.wikipedia.org/wiki/Graphical_model):

A graphical model or probabilistic graphical model (PGM) is a probabilistic model for which a graph expresses the conditional dependence structure between random variables. They are commonly used in probability theory, statistics—particularly Bayesian statistics—and machine learning.

In that case, would it be "Bayesian"? What about something like Conditional Random Field? Is that Bayesian as well?

Answer 1

Probabilistic Graphical Models (PGMs) are:

• Connectionist: RBMs are PGMs and neural networks (source)
• Bayesian: Bayes Networks are bayesian (Wikipedia article)
• Symbolist: Markov Logic Networks (source)
• Analogizers and Evolutionaries: According to Domingos, they are also in Markov Logic Networks.

So the answer is that you can't simply categorize such a general technique as probabilistic graphical models in a single one of those categories.

See also: "The Five Tribes of Machine Learning (And What You Can Learn from Each)," Pedro Domingos

Answer 2

Only Domingos knows for sure, since he invented this taxonomy, but I'd guess it would fall under "connectionists" (which he associates with neural networks), since graphs are all about connections (between random variables). Bayesians would be my second choice. CRFs are not natively Bayesian (you don't use priors or posteriors of the model parameters), but they can be augmented into one.

Answer 3

Based on this paper (Qi, Szummer, and Minka. 2005. Bayesian Conditional Random Fields):

In this paper, we propose Bayesian Conditional Random Fields (BCRF), a novel Bayesian approach to training and inference for conditional random fields

We can see that the original CRF model is presumably not Bayesian, since this paper's contribution is a novel Bayesian approach to the CRF model.

Based on this excerpt of the book:

Each of the five tribes of machine learning has its own master algorithm, a general-purpose learner that you can in principle use to discover knowledge from data in any domain. The symbolists’ master algorithm is inverse deduction, the connectionists’ is backpropagation, the evolutionaries’ is genetic programming, the Bayesians’ is Bayesian inference, and the analogizers’ is the support vector machine.

It seems that CRF can be put into connectionists (since CRF calculates optimizes a function based on the gradients) or analogizers (since CRF uses "soft-examples" calculated by softmax to separate the good output and bad outputs, as compared to the support vectors in (structured) SVM calculated by max. For example, this paper shows that the difference CRF and SSVM are the addition of cost function and change from softmax to max)