For example I have the following data structure:

user: Chris
age: 32
income: 60.000
basket value: 45

I want predict the basket value, and my features are the age and income.

With a linear regression I get a regression function as the result of the fitting for example: $$y = 0.5x + 0.785$$

Now I can use the function for prediction.

What is the form of the result of the fitting by regression decision tree? Is it also a function?

  • $\begingroup$ Usually a piece-wise constant function. $\endgroup$
    – Emre
    Oct 30 '17 at 18:12

Yes. It is also a function, but not an affine transformation of the input but a relatively complex sum of products of indicator functions of the input. Usually, this function is represented by the fitted tree and not as a formula.

So e.g. if you learn a tree of depth one and the split is at age 40 with mean response of 80 if age < 40 and mean response of 100 if age $\ge$ 40, then the function could look like $$ \hat f(\text{age}, \text{income}) = 80 \cdot {\mathbf 1}\{\text{age} < 40\} + 100 \cdot {\mathbf 1}\{\text{age} \ge 40\} $$ You can maybe imagine how long the formula is if the depth is 7...

  • $\begingroup$ Thats intristing. Can you give me a simple example function of this for my understanding? And how the prediction works when the function is represented by the fittes tree? $\endgroup$ Oct 30 '17 at 19:31

Decision trees solve different types questions.

E.g. will someone with Age A, Gender G, Income I buy product X or Y.

There is no fitting involved, but the model might internally calculate/come up with an Age boundary or Income boundary that would switch the decision from X to Y.

  • $\begingroup$ Your example is a classification problem but, right? I‘m searching for a regression tree. $\endgroup$ Oct 29 '17 at 9:37
  • $\begingroup$ regression trees are constructed in a similar fashion to classification trees. the difference is that for regression, you assign to each leaf average of the target values that fall within that leaf, whereas in classification, you'd assign the majority label instead. $\endgroup$
    – darXider
    Oct 30 '17 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.