I am new to decision tree method. For decision tree regression model, does it just fit a piece wise step function over data? When and why would people prefer it over some traditional regression like locally weighted (LOESS) regression?
There are two important differences between decision trees and regression:
Decision tree fit a straight a line (mean of the dependent variable for the feature space). Regression fits a sloped line (rise over run).
Decision trees typically do not predict values outside the observed range. Linear regression can predict values outside the observed range.