13
$\begingroup$

So we have potential for a machine learning application that fits fairly neatly into the traditional problem domain solved by classifiers, i.e., we have a set of attributes describing an item and a "bucket" that they end up in. However, rather than create models of probabilities like in Naive Bayes or similar classifiers, we want our output to be a set of roughly human-readable rules that can be reviewed and modified by an end user.

Association rule learning looks like the family of algorithms that solves this type of problem, but these algorithms seem to focus on identifying common combinations of features and don't include the concept of a final bucket that those features might point to. For example, our data set looks something like this:

Item A { 4-door, small, steel } => { sedan }
Item B { 2-door, big,   steel } => { truck }
Item C { 2-door, small, steel } => { coupe }

I just want the rules that say "if it's big and a 2-door, it's a truck," not the rules that say "if it's a 4-door it's also small."

One workaround I can think of is to simply use association rule learning algorithms and ignore the rules that don't involve an end bucket, but that seems a bit hacky. Have I missed some family of algorithms out there? Or perhaps I'm approaching the problem incorrectly to begin with?

Improve this question
$\endgroup$
10
$\begingroup$

C45 made by Quinlan is able to produce rule for prediction. Check this Wikipedia page. I know that in Weka its name is J48. I have no idea which are implementations in R or Python. Anyway, from this kind of decision tree you should be able to infer rules for prediction.

Later edit

Also you might be interested in algorithms for directly inferring rules for classification. RIPPER is one, which again in Weka it received a different name JRip. See the original paper for RIPPER: Fast Effective Rule Induction, W.W. Cohen 1995

Improve this answer
$\endgroup$
3
  • $\begingroup$ I had experimented with C45/J48 in a previous project. I did not realize there were rules I could retrieve from it. I'll also check out RIPPER. Thanks! $\endgroup$ – super_seabass May 27 '14 at 13:53
  • $\begingroup$ Also check out the C50 package in R. $\endgroup$ – nfmcclure Jun 6 '14 at 14:56
  • $\begingroup$ Wanted to provide an update for this question/answer: we've been using JRip with some success, but our new leading contender is FURIA (cs.uni-paderborn.de/fileadmin/Informatik/eim-i-is/PDFs/…). It's generating the best rules for human review/use because it tries to generate an exhaustive ruleset. JRip makes nice rules, but it has a "default" rule for classification when no other rules apply. Default buckets don't work well in our project's business context, we need exhaustive rules. $\endgroup$ – super_seabass Oct 21 '14 at 17:36
8
$\begingroup$

It's actually even simpler than that, from what you describe---you're just looking for a basic classification tree algorithm (so no need for slightly more complex variants like C4.5 which are optimized for prediction accuracy). The canonical text is this.

This is readily implemented in R and Python.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I do not agree trees are of help here. It is a matter of filtering rules, and that can be achieved with the arules package in R. $\endgroup$ – adesantos Jun 25 '14 at 10:13
3
$\begingroup$

You could take a look at CN2 rule learner in Orange 2

Improve this answer
$\endgroup$
1
$\begingroup$

You should try arules package in R. It allows you to create not only the association rules but also to specify the length of each rule, the importance of each rule and also you can filter them, which is what you are looking for (try the rhs() command of this package).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.