3
$\begingroup$

I am new to decision tree method. For decision tree regression model, does it just fit a piece wise step function over data? When and why would people prefer it over some traditional regression like locally weighted (LOESS) regression?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Yes, decision tree regression outputs are traditionally locally linear, but you can create a model that behaves differently! LOESS is closer to k-NN regression, and shares its strengths/weaknesses. For example, k-NN requires computation of the nearest neighbors, and therefore the storage of the data set for querying. Decision trees are more memory efficient, but only produce locally linear outputs, etc. $\endgroup$
    – Emre
    Oct 27, 2017 at 18:40

1 Answer 1

Reset to default
0
$\begingroup$

There are two important differences between decision trees and regression:

  1. Decision tree fit a straight a line (mean of the dependent variable for the feature space). Regression fits a sloped line (rise over run).

  2. Decision trees typically do not predict values outside the observed range. Linear regression can predict values outside the observed range.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.