# How do I perform Naive Bayes Classification with a Bayesian Belief Network?

I've been writing a java library that I want to use to build Bayesian Belief Networks. I have classes that I use to build a Directed Graph

public class Node{
private String label;
private List<Node> adjacencyList = new ArrayList<Node>();
private Frequency<String> distribution = new Frequency<String>();

public String getLabel() {
return label;
}

public void setLabel(String label) {
this.label = label;
}

public List<Node> getAdjacencyList(){
return adjacencyList;
}

public void addNeighbour(Node neighbour){
adjacencyList.add(neighbour);
}

public void setDistribution(List<String> data){

for(String s:data){
distribution.addValue(s);
}
}

public double getDistributionValue(String value){
return distribution.getPct(value);
}
}


Graph

public class DirectedGraph {

Map<String,Node> graph = new HashMap<String,Node>();

public void addVertex(String label){
Node vertex = new Node();
vertex.setLabel(label);
graph.put(label, vertex);
}

public void addEdge(String here, String there){
Node nHere = graph.get(here);
Node nThere = graph.get(there);
nThere.addNeighbour(nHere);
graph.put(there,nThere);
}

public List<Node> getNeighbors(String vertex){
return graph.get(vertex).getAdjacencyList();
}

public int degree(String vertex){
return graph.get(vertex).getAdjacencyList().size();
}

public boolean hasVertex(String vertex){
return graph.containsKey(vertex);
}

public boolean hasEdge(String here, String there){
Set<Node> nThere = new HashSet<Node>(graph.get(there).getAdjacencyList());
boolean thereConHere = nThere.contains(here);
return (thereConHere);
}
}


I have a class that I use to keep track of the probability distribution of a data set

public class Frequency<T extends Comparable<T>> {

private Multiset event = HashMultiset.create();
private Multimap event2 = LinkedListMultimap.create();

public void addValue(T data){
if(event2.containsKey(data) == false){
event2.put(data,data);
}

event.add(data);
}

public void clear(){

this.event = null;
this.event2 = null;

this.event = HashMultiset.create();
this.event2 = LinkedListMultimap.create();
}

public double getPct(T data){
int numberOfIndElements = event.count(data);
int totalNumOfElements = event.size();
return (double) numberOfIndElements/totalNumOfElements;
}

public int getNum(T data){
int numberOfIndElements = event.count(data);
return numberOfIndElements;
}

public int getSumFreq(){
return event.size();
}

public int getUniqueCount(){
return event.entrySet().size();
}

public String[] getKeys(){
Set<String> test = event2.keySet();
Object[] keys = test.toArray();
String[] keysAsStrings = new String[keys.length];

for(int i=0;i<keys.length;i++){
keysAsStrings[i] = (String) keys[i];
}

return keysAsStrings;
}
}


as well as another function that I can use to calculate conditional probabilities

public double conditionalProbability(List<String> interestedSet,
List<String> reducingSet,
String interestedClass,
String reducingClass){
List<Integer> conditionalData = new LinkedList<Integer>();
double returnProb = 0;
iFrequency.clear();
rFrequency.clear();

this.setInterestedFrequency(interestedSet);
this.setReducingFrequency(reducingSet);

for(int i = 0;i<reducingSet.size();i++){
if(reducingSet.get(i).equalsIgnoreCase(reducingClass)){
if(interestedSet.get(i).equalsIgnoreCase(interestedClass)){
conditionalData.add(i);
}
}
}

int numerator = conditionalData.size();
int denominator = this.rFrequency.getNum(reducingClass);

if(denominator !=0){
returnProb = (double)numerator/denominator;
}

iFrequency.clear();
rFrequency.clear();
return returnProb;
}


However, I'm still not sure how to hook everything up in order to perform classification.

I was reading over a paper entitled Comparing Bayesian Network Classifiers to try and get an understanding.

Let's say that I am trying to predict a person's sex based on the attributes height, weight and shoe size. My understanding is that I would have Sex as my parent/classification node and height, weight and shoe size would by my child nodes.

This is what I'm confused about. The various classification nodes only keep track of the probability distribution of their respective attributes, but I'd need the conditional probabilities in order to perform classification.

I have an older version of Naive Bayes that I wrote

public void naiveBayes(Data data,List<String> targetClass, BayesOption bayesOption,boolean headers){
//intialize variables
int numOfClasses = data.getNumOfKeys();//.getHeaders().size();
String[] keyNames = data.getKeys();//  data.getHeaders().toArray();
double conditionalProb = 1.0;
double prob = 1.0;
String[] rClass;
String priorName;

iFrequency.clear();
rFrequency.clear();

if(bayesOption.compareTo(BayesOption.TRAIN) == 0){
this.setInterestedFrequency(targetClass);
this.targetClassKeys = Util.convertToStringArray(iFrequency.getKeys());

for(int i=0;i<this.targetClassKeys.length;i++){
priors.put(this.targetClassKeys[i],iFrequency.getPct(this.targetClassKeys[i]));
}

}

//for each classification in the target class
for(int i=0;i<this.targetClassKeys.length;i++){

//get all of the different classes for that variable
for(int j=0;j<numOfClasses;j++){

String reducingKey = Util.convertToString(keyNames[j]);
List<String> reducingClass = data.dataColumn(reducingKey,DataOption.GET,true);// new ArrayList(data.getData().get(reducingKey));
this.setReducingFrequency(reducingClass);
Object[] reducingClassKeys = rFrequency.getKeys();
rClass = Util.convertToStringArray(reducingClassKeys);

for(int k=0;k<reducingClassKeys.length;k++){

if(bayesOption.compareTo(BayesOption.TRAIN) == 0){
conditionalProb = conditionalProbability(targetClass, reducingClass, this.targetClassKeys[i], rClass[k]);
priorName = this.targetClassKeys[i]+"|"+rClass[k];
priors.put(priorName,conditionalProb);
}

if(bayesOption.compareTo(BayesOption.PREDICT) == 0){
priorName = this.targetClassKeys[i]+"|"+rClass[k];
prob = prob * priors.get(priorName);

}
}
rFrequency.clear();

}

if(BayesOption.PREDICT.compareTo(bayesOption) == 0){
prob = prob * priors.get(this.targetClassKeys[i]);
Pair<String,Double> pred = new Pair<String, Double>(this.targetClassKeys[i],prob);
this.predictions.add(pred);
}

}

this.iFrequency.clear();
this.rFrequency.clear();

}


So I generally understand how the math works, but I'm not quite sure how I'm supposed to get things to work with this specific architecture.

How do I calculate the conditional probabilities?

Can somebody explain this discrepancy to me please?

## 2 Answers

After reading some more papers, I realize that I misunderstood how the graphs work. The graphs are supposed to contain the conditional probabilities based on their parent(s).

This clears up the doubts that I had before.

For more information, see this book chapter.

I think , if you are not in a particular interest to use Java(which is not a deditcated language for machine learning) you could find a lot of references in R and Python: https://pymc-devs.github.io/pymc/ http://www.bayespy.org/index.html http://www.bnlearn.com/examples/

• I want to use java specifically. – j.jerrod.taylor Dec 4 '15 at 18:32