I am new to decision tree method. For decision tree regression model, does it just fit a piece wise step function over data? When and why would people prefer it over some traditional regression like locally weighted (LOESS) regression?

  • 2
    $\begingroup$ Yes, decision tree regression outputs are traditionally locally linear, but you can create a model that behaves differently! LOESS is closer to k-NN regression, and shares its strengths/weaknesses. For example, k-NN requires computation of the nearest neighbors, and therefore the storage of the data set for querying. Decision trees are more memory efficient, but only produce locally linear outputs, etc. $\endgroup$
    – Emre
    Oct 27 '17 at 18:40

There are two important differences between decision trees and regression:

  1. Decision tree fit a straight a line (mean of the dependent variable for the feature space). Regression fits a sloped line (rise over run).

  2. Decision trees typically do not predict values outside the observed range. Linear regression can predict values outside the observed range.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.