1
$\begingroup$

I was looking to learn about Bayesian theory in decision tree and how it avoids overfitting but couldn't find any tutorials for someone just starting. Do you know any resources to learn about it?

$\endgroup$
  • $\begingroup$ This is too open-ended to be a good question. Instead of asking for recommendations, can you specify more about what you want to know, what you know so far, and what your question is? $\endgroup$ – Sean Owen Dec 23 '14 at 15:41
1
$\begingroup$

Regarding the being new to decision trees and wanting to get off the ground, I wrote a tutorial on decision trees that will help.

Regarding methods to avoid overfitting: The game for any model is to limit its complexity to what is reasonable given the data you have. Complexity in decision trees is manifested as adding new decision boundaries, so any limit in complexity is a limit in the decision boundaries it can draw. Two common ways to do this is to place constraints on when a new decision can be created (a minimum of data in a leaf, significant increase in information, etc) or more simply to limit the max depth of the tree.

$\endgroup$
0
$\begingroup$

A good method to reduce overfitting in Decision Trees is Pruning, i.e. not to generate the tree downto the last attribute (pre-prune), or generate the "full" tree and then post-prune. Not sure if it pertains to Bayesian theory per se, but the leaf nodes of the "pruned" tree have probabilities about the various classifications that are constrained by the pruned branches. There is an implementation in Weka (J48 classifier) and the accompanying book Data Mining - Practical Machine Learning Tools and Techniques:

http://www.amazon.com/Data-Mining-Practical-Techniques-Management/dp/0120884070

If you would like to read in more detail on what I just wrote above, it is explained in detail in Chapter 6.1

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.