13
$\begingroup$

I have two questions related to decision trees:

  1. If we have a continuous attribute, how do we choose the splitting value?

    Example: Age=(20,29,50,40....)

  2. Imagine that we have a continuous attribute $f$ that have values in $R$. How can I write an algorithm that finds the split point $v$, in order that when we split $f$ by $v$, we have a minimum gain for $f>v$?

$\endgroup$
17
$\begingroup$

In order to come up with a split point, the values are sorted, and the mid-points between adjacent values are evaluated in terms of some metric, usually information gain or gini impurity. For your example, lets say we have four examples and the values of the age variable are $(20, 29, 40, 50)$. The midpoints between the values $(24.5, 34.5, 45)$ are evaluated, and whichever split gives the best information gain (or whatever metric you're using) on the training data is used.

You can save some computation time by only checking split points that lie between examples of different classes, because only these splits can be optimal for information gain.

$\endgroup$
  • $\begingroup$ @timleathart the OP is expecting to be "spoon fed" the implementation in R. I wonder what the OP has tried so far with reference to R implementation? How about "showing some effort", OP? $\endgroup$ – mnm Nov 4 '17 at 0:31
  • $\begingroup$ @timleathart but normaly for an attribut f we choose the split v that give the biggest information gain for f>v, but here look at the question they asked for a minimum gain. $\endgroup$ – WALID BELRHALMIA Nov 4 '17 at 9:35
  • $\begingroup$ @timleathart, Can you explain more ? I need to know best optimized way of identifying such splits and check for information gain. Lets say one variable has lot of variation and other is almost constant. How many such splits should be there? $\endgroup$ – Arpit Sisodia Jul 3 at 11:32
  • $\begingroup$ @timeleathart, extending ur answer, this split will not be optimized when values are (20,21,22,23, 45,67,80). shouldn't min to max iteration can be used here? Please correct me if i am wrong in my assumption:) $\endgroup$ – Arpit Sisodia Jul 3 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.