For example I have the following data structure:

user: Chris
age: 32
income: 60.000
basket value: 45

I want predict the basket value, and my features are the age and income.

With a linear regression I get a regression function as the result of the fitting for example: $$y = 0.5x + 0.785$$

Now I can use the function for prediction.

What is the form of the result of the fitting by regression decision tree? Is it also a function?

  • $\begingroup$ Usually a piece-wise constant function. $\endgroup$
    – Emre
    Oct 30, 2017 at 18:12

2 Answers 2


Yes. It is also a function, but not an affine transformation of the input but a relatively complex sum of products of indicator functions of the input. Usually, this function is represented by the fitted tree and not as a formula.

So e.g. if you learn a tree of depth one and the split is at age 40 with mean response of 80 if age < 40 and mean response of 100 if age $\ge$ 40, then the function could look like $$ \hat f(\text{age}, \text{income}) = 80 \cdot {\mathbf 1}\{\text{age} < 40\} + 100 \cdot {\mathbf 1}\{\text{age} \ge 40\} $$ You can maybe imagine how long the formula is if the depth is 7...

  • $\begingroup$ Thats intristing. Can you give me a simple example function of this for my understanding? And how the prediction works when the function is represented by the fittes tree? $\endgroup$ Oct 30, 2017 at 19:31
  • 1
    $\begingroup$ dbplyr will convert a 1-tree random forest into a SQL query that is mostly equation. It is both beautiful and scary. Dangerous in the wrong hands. $\endgroup$ Nov 12, 2021 at 18:41
  • 1
    $\begingroup$ @EngrStudent: I will try that! With the right hands, hopefully. $\endgroup$
    – Michael M
    Nov 12, 2021 at 19:11

Decision trees solve different types questions.

E.g. will someone with Age A, Gender G, Income I buy product X or Y.

There is no fitting involved, but the model might internally calculate/come up with an Age boundary or Income boundary that would switch the decision from X to Y.

  • $\begingroup$ Your example is a classification problem but, right? I‘m searching for a regression tree. $\endgroup$ Oct 29, 2017 at 9:37
  • $\begingroup$ regression trees are constructed in a similar fashion to classification trees. the difference is that for regression, you assign to each leaf average of the target values that fall within that leaf, whereas in classification, you'd assign the majority label instead. $\endgroup$
    – darXider
    Oct 30, 2017 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.