# Decision trees vs Oblique decision trees

What are oblique decision trees ?

What are the differences between them and classic decision trees ?

An oblique decision tree is a decision tree in which the conditions used to split the data put a constraint over a linear combination of the features.

Say that you have a dependent variable $$Y$$ that you wish to explain using the features $$X_1, \dots, X_d$$. In a typical decision tree, the nodes will only check the value of a single feature (e.g. $$X_1 > \mu_{X_1}$$) whereas in an oblique decision tree, the nodes will check whether a linear combination of a number of features is smaller or larger than a certain quantity (e.g. $$2X_1 + 3X_2 > \mu_{X_1, X_2}$$)

The term used to talk about decision trees that only check the value of a single feature per node is axis-aligned decision trees. This is because each of the hyperplanes used to perform the splitting is perpendicular to a certain axis/feature. For instance, the hyperplane that splits the data points into those fulfilling the condition $$X_1 > \mu_1$$ and those who don't will be defined by the equation $$(E): X_1 = \mu_1$$ and will be perpendicular to the axis $$X_1$$.

In general, an oblique condition can be expressed as follows: $$\sum_{i=1}^{d}w_iX_i \space + \space w_{d+1} \space < \space 0$$

where $$w_1, \dots, w_{d+1}$$ are real numbers representing the weights of each feature.

As you could have probably guessed, axis-aligned decision trees can be thought of as a special case of oblique decision trees in which all the weights $$(w_i)_{i \in [1,d]}$$ are equal to $$0$$ except for one.

A major difference between axis-aligned decision trees and oblique decision trees lies within the possible number of conditions on which to perform the split. In the former, the number of all possible conditions/splits is at most $$d.n$$ where $$d$$ is the number of dimensions and $$n$$ is the number of data points. In the latter, with the weights being real numbers, there is an infinite number of possible splits to pick from. This difference plays a key role in defining the kind of algorithms used to build each type of these decision trees.

For further reference:

OC1: A randomized algorithm for building oblique decision trees.

A System for Induction of Oblique Decision Trees

Inducing Oblique Decision Trees With Evolutionary Algorithms