I was looking to learn about Bayesian theory in decision tree and how it avoids overfitting but couldn't find any tutorials for someone just starting. Do you know any resources to learn about it?
Regarding the being new to decision trees and wanting to get off the ground, I wrote a tutorial on decision trees that will help.
Regarding methods to avoid overfitting: The game for any model is to limit its complexity to what is reasonable given the data you have. Complexity in decision trees is manifested as adding new decision boundaries, so any limit in complexity is a limit in the decision boundaries it can draw. Two common ways to do this is to place constraints on when a new decision can be created (a minimum of data in a leaf, significant increase in information, etc) or more simply to limit the max depth of the tree.
A good method to reduce overfitting in Decision Trees is Pruning, i.e. not to generate the tree downto the last attribute (pre-prune), or generate the "full" tree and then post-prune. Not sure if it pertains to Bayesian theory per se, but the leaf nodes of the "pruned" tree have probabilities about the various classifications that are constrained by the pruned branches. There is an implementation in Weka (J48 classifier) and the accompanying book Data Mining - Practical Machine Learning Tools and Techniques:
If you would like to read in more detail on what I just wrote above, it is explained in detail in Chapter 6.1
I suggest the Balaji Lakshminarayanan's website for his work on Mondrian forests and Antonio Linero's website on this topic. Another related topic is the Bayesian multiple adaptive regression trees. And the below paper also summarizes some Bayesian methods on tree-based modes, - https://www.intechopen.com/books/enhanced-expert-systems/classic-and-bayesian-tree-based-methods