1
$\begingroup$

For example I have the following data structure:

user: Chris
age: 32
income: 60.000
basket value: 45

I want predict the basket value, and my features are the age and income.

With a linear regression I get a regression function as the result of the fitting for example: $$y = 0.5x + 0.785$$

Now I can use the function for prediction.

What is the form of the result of the fitting by regression decision tree? Is it also a function?

Improve this question
$\endgroup$
1
  • $\begingroup$ Usually a piece-wise constant function. $\endgroup$
    – Emre
    Oct 30, 2017 at 18:12

2 Answers 2

Reset to default
6
$\begingroup$

Yes. It is also a function, but not an affine transformation of the input but a relatively complex sum of products of indicator functions of the input. Usually, this function is represented by the fitted tree and not as a formula.

So e.g. if you learn a tree of depth one and the split is at age 40 with mean response of 80 if age < 40 and mean response of 100 if age $\ge$ 40, then the function could look like $$ \hat f(\text{age}, \text{income}) = 80 \cdot {\mathbf 1}\{\text{age} < 40\} + 100 \cdot {\mathbf 1}\{\text{age} \ge 40\} $$ You can maybe imagine how long the formula is if the depth is 7...

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thats intristing. Can you give me a simple example function of this for my understanding? And how the prediction works when the function is represented by the fittes tree? $\endgroup$
    – user43348044
    Oct 30, 2017 at 19:31
  • 1
    $\begingroup$ dbplyr will convert a 1-tree random forest into a SQL query that is mostly equation. It is both beautiful and scary. Dangerous in the wrong hands. $\endgroup$
    – EngrStudent
    Nov 12, 2021 at 18:41
  • 1
    $\begingroup$ @EngrStudent: I will try that! With the right hands, hopefully. $\endgroup$
    – Michael M
    Nov 12, 2021 at 19:11
0
$\begingroup$

Decision trees solve different types questions.

E.g. will someone with Age A, Gender G, Income I buy product X or Y.

There is no fitting involved, but the model might internally calculate/come up with an Age boundary or Income boundary that would switch the decision from X to Y.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Your example is a classification problem but, right? I‘m searching for a regression tree. $\endgroup$
    – user43348044
    Oct 29, 2017 at 9:37
  • $\begingroup$ regression trees are constructed in a similar fashion to classification trees. the difference is that for regression, you assign to each leaf average of the target values that fall within that leaf, whereas in classification, you'd assign the majority label instead. $\endgroup$
    – darXider
    Oct 30, 2017 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.