For classification, it is obvious how a decision tree is used to make a prediction.You just have to find the final leaf. However for regression problems, how can you find the prediction considering the continous aspect of the variable to predict?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
6
Depends on the implementation but commonly used is a certain cursive partitioning method called CART. The algorithm works as following: It searches for every distinct values for your predictors and chooses to the split based on what minimize the SSE for two groups of dependent variables. The difference is within the SSE is usually the difference between the actual value and the average of the sample or the difference between the actual value and the output of a linear regression. For each group, the method will recursively split the predictor values within the groups.
In practice, the method stops when a certain sample size threshold is met. The node with the lowest SSE becomes the root node.
Reference: You can read more about tree based regressions here: Tree-based regressions
-
-
$\begingroup$ So depending on the stop criterion, there are two possibilites? Either stop at the current node and average the values in the node leaves or continue the split? $\endgroup$ Jan 1, 2018 at 12:02
-
$\begingroup$ Doing an example is a bit tedious to make up and write. Here's a brief overview. 1 Start with a single node with all points, calculate the average and SSE. 2. If all points have the same value for an input variable stop. Else, search over all binary splits of all variables for the one that makes the lowest SSE. If the largest decrease in SSE is else than a threshold or a node has less than q points stop. Otherwise split again and make two nodes. 3. Go back to step 1 for each new node. $\endgroup$– TophatJan 1, 2018 at 15:14
-
$\begingroup$ Just add a reference and this will be The answer! $\endgroup$ Jan 2, 2018 at 8:39
-
$\begingroup$
$\endgroup$
First and foremost you need to know the difference between the type of data you are trying to predict. The two general categories are discrete and continuous. Most people tend to miss out that classification is at its core a discrete "regression". The values are predicted for discrete variables by considering a decision-based approach which ultimately leads to finding or predicting the variable in question. You can call this classification.
For continuous variables, however, since the values can have a certain range it most closely mirrors a curve fit with the addition of an error boundary. I think this should clarify what you're essentially trying to ask.
