Can a decision in a node of a decision tree be based on comparison between 2 columns of the dataset?

Question

Assume the features in the dataframe are columns - A,B,C and my target is Y

Can my decision tree have a decision node which looks for say, if A>B then true else false?

Answer 1

Yes, but not in any implementation that I am aware of.

The idea is mentioned in Elements of Statistical Learning, near the end of section 9.2.4 under the heading "Linear Combination Splits." But this is not implemented in the popular CART or Quinlan-family of decision tree algorithms*, it is not done in sklearn's trees, and I do not know of any other python or R package that use it.

Some R packages do a more limited version, where a split can be made on two features, but these splits are of the form "$x_1>\alpha\text{ and }x_2\leq\beta$" as opposed to direct comparisons of the variables. See https://stats.stackexchange.com/questions/4356/does-rpart-use-multivariate-splits-by-default

An obvious problem is computational requirements: just checking over all pairs of features is now quadratic, and allowing arbitrary linear combinations of two features is potentially much much larger. On the other hand, if you want to restrict to direct comparisons $x_1\geq x_2$ (with no coefficients), that should be tractable (if substantially slower than CART). The Elements authors suggests Hierarchical Mixtures of Experts model instead if incorporating linear combinations is desired.

Oh, one more comment. If you really want splits like $x_1\geq x_2$, you could just generate all the features $x_i-x_j$; then a more common implementation of decision trees will be able to make your splits, when considering these new features. (Probably there will be some side effects, and still the computational problem arises: you've added $\binom{m}{2}$ features.)

---

* I've found a comment that suggests that CART does support multi-feature ("surrogate") splits?:
https://stackoverflow.com/a/9996741/10495893