I am new to decision tree method. For decision tree regression model, does it just fit a piece wise step function over data? When and why would people prefer it over some traditional regression like locally weighted (LOESS) regression?
1
-
2$\begingroup$ Yes, decision tree regression outputs are traditionally locally linear, but you can create a model that behaves differently! LOESS is closer to k-NN regression, and shares its strengths/weaknesses. For example, k-NN requires computation of the nearest neighbors, and therefore the storage of the data set for querying. Decision trees are more memory efficient, but only produce locally linear outputs, etc. $\endgroup$– EmreOct 27, 2017 at 18:40
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
There are two important differences between decision trees and regression:
Decision tree fit a straight a line (mean of the dependent variable for the feature space). Regression fits a sloped line (rise over run).
Decision trees typically do not predict values outside the observed range. Linear regression can predict values outside the observed range.
answered Nov 29, 2020 at 3:54