# Forecasting: How Decision Tree work?

For example I have the following data structure:

user: Chris
age: 32
income: 60.000


I want predict the basket value, and my features are the age and income.

With a linear regression I get a regression function as the result of the fitting for example: $$y = 0.5x + 0.785$$

Now I can use the function for prediction.

What is the form of the result of the fitting by regression decision tree? Is it also a function?

• Usually a piece-wise constant function.
– Emre
Commented Oct 30, 2017 at 18:12

Yes. It is also a function, but not an affine transformation of the input but a relatively complex sum of products of indicator functions of the input. Usually, this function is represented by the fitted tree and not as a formula.

So e.g. if you learn a tree of depth one and the split is at age 40 with mean response of 80 if age < 40 and mean response of 100 if age $\ge$ 40, then the function could look like $$\hat f(\text{age}, \text{income}) = 80 \cdot {\mathbf 1}\{\text{age} < 40\} + 100 \cdot {\mathbf 1}\{\text{age} \ge 40\}$$ You can maybe imagine how long the formula is if the depth is 7...

• Thats intristing. Can you give me a simple example function of this for my understanding? And how the prediction works when the function is represented by the fittes tree? Commented Oct 30, 2017 at 19:31
• dbplyr will convert a 1-tree random forest into a SQL query that is mostly equation. It is both beautiful and scary. Dangerous in the wrong hands. Commented Nov 12, 2021 at 18:41
• @EngrStudent: I will try that! With the right hands, hopefully. Commented Nov 12, 2021 at 19:11

Decision trees solve different types questions.

E.g. will someone with Age A, Gender G, Income I buy product X or Y.

There is no fitting involved, but the model might internally calculate/come up with an Age boundary or Income boundary that would switch the decision from X to Y.

• Your example is a classification problem but, right? I‘m searching for a regression tree. Commented Oct 29, 2017 at 9:37
• regression trees are constructed in a similar fashion to classification trees. the difference is that for regression, you assign to each leaf average of the target values that fall within that leaf, whereas in classification, you'd assign the majority label instead. Commented Oct 30, 2017 at 19:10