# Spark Mllib - FPG-Growth - Machine Learning

Is the FPG-Growth an Machine Learning algorithm? Because I'm looking at this code:

import org.apache.spark.mllib.fpm.FPGrowth
import org.apache.spark.rdd.RDD

val data = sc.textFile("data/mllib/sample_fpgrowth.txt")

val transactions: RDD[Array[String]] = data.map(s => s.trim.split(' '))

val fpg = new FPGrowth()
.setMinSupport(0.2)
.setNumPartitions(10)
val model = fpg.run(transactions)

model.freqItemsets.collect().foreach { itemset =>
println(itemset.items.mkString("[", ",", "]") + ", " + itemset.freq)
}

val minConfidence = 0.8
model.generateAssociationRules(minConfidence).collect().foreach { rule =>
println(
rule.antecedent.mkString("[", ",", "]")
+ " => " + rule.consequent .mkString("[", ",", "]")
+ ", " + rule.confidence)
}


And I'm not seeing how the algoritm can learn because it don't have an train, test or validation set...

Thanks

FP-growth is a frequent pattern association rule learning algorithm. Thus it's a rule based machine learning algorithm.

When you call the following :

val model = fpg.run(transactions)


You are actually creating and Frequent Pattern model without generating candidates. So actually you'll need to generate afterwards if you need to use them with :

model.generateAssociationRules(minConfidence)


Now, concerning the usual flow of building and validating such models, your code doesn't deal with that. It's usually done through quality measures variations.

It can also be done through Feature extraction techniques.

With such techniques, you should consider the following. For each association rule, you'll need to measure the improvements in accuracy that a commonly used predictor can obtain from an additional feature, constructed according to the exceptions to the rule. In other terms, you'll have to select a reference set of rules that should help your model perform better. I strongly advice you to read this paper about the topic.

So now what does that mean ? This means that you'll need to implement that pipeline yourself because it's not implemented in Spark yet.