I have two questions related to decision trees:

  1. If we have a continuous attribute, how do we choose the splitting value?

    Example: Age=(20,29,50,40....)

  2. Imagine that we have a continuous attribute $f$ that have values in $R$. How can I write an algorithm that finds the split point $v$, in order that when we split $f$ by $v$, we have a minimum gain for $f>v$?


1 Answer 1


In order to come up with a split point, the values are sorted, and the mid-points between adjacent values are evaluated in terms of some metric, usually information gain or gini impurity. For your example, lets say we have four examples and the values of the age variable are $(20, 29, 40, 50)$. The midpoints between the values $(24.5, 34.5, 45)$ are evaluated, and whichever split gives the best information gain (or whatever metric you're using) on the training data is used.

You can save some computation time by only checking split points that lie between examples of different classes, because only these splits can be optimal for information gain.

  • $\begingroup$ @timleathart but normaly for an attribut f we choose the split v that give the biggest information gain for f>v, but here look at the question they asked for a minimum gain. $\endgroup$ Commented Nov 4, 2017 at 9:35
  • $\begingroup$ @timleathart, Can you explain more ? I need to know best optimized way of identifying such splits and check for information gain. Lets say one variable has lot of variation and other is almost constant. How many such splits should be there? $\endgroup$ Commented Jul 3, 2019 at 11:32
  • $\begingroup$ @timeleathart, extending ur answer, this split will not be optimized when values are (20,21,22,23, 45,67,80). shouldn't min to max iteration can be used here? Please correct me if i am wrong in my assumption:) $\endgroup$ Commented Jul 3, 2019 at 11:35
  • $\begingroup$ This clarifies my confusions! $\endgroup$ Commented Nov 16, 2019 at 12:10
  • 1
    $\begingroup$ I think the use of midpoints is an implementation detail; IIRC, some may split as <= each value itself. And there are additional short-cuts for efficiency, e.g. histogram binning. $\endgroup$
    – Ben Reiniger
    Commented May 13, 2020 at 19:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.