2
$\begingroup$

What are oblique decision trees ?

What are the differences between them and classic decision trees ?

$\endgroup$
0

1 Answer 1

1
$\begingroup$

An oblique decision tree is a decision tree in which the conditions used to split the data put a constraint over a linear combination of the features.

Say that you have a dependent variable $Y$ that you wish to explain using the features $X_1, \dots, X_d$. In a typical decision tree, the nodes will only check the value of a single feature (e.g. $X_1 > \mu_{X_1}$) whereas in an oblique decision tree, the nodes will check whether a linear combination of a number of features is smaller or larger than a certain quantity (e.g. $2X_1 + 3X_2 > \mu_{X_1, X_2}$)

The term used to talk about decision trees that only check the value of a single feature per node is axis-aligned decision trees. This is because each of the hyperplanes used to perform the splitting is perpendicular to a certain axis/feature. For instance, the hyperplane that splits the data points into those fulfilling the condition $X_1 > \mu_1$ and those who don't will be defined by the equation $(E): X_1 = \mu_1$ and will be perpendicular to the axis $X_1$.

In general, an oblique condition can be expressed as follows: $$ \sum_{i=1}^{d}w_iX_i \space + \space w_{d+1} \space < \space 0 $$

where $w_1, \dots, w_{d+1}$ are real numbers representing the weights of each feature.

As you could have probably guessed, axis-aligned decision trees can be thought of as a special case of oblique decision trees in which all the weights $(w_i)_{i \in [1,d]}$ are equal to $0$ except for one.

A major difference between axis-aligned decision trees and oblique decision trees lies within the possible number of conditions on which to perform the split. In the former, the number of all possible conditions/splits is at most $d.n$ where $d$ is the number of dimensions and $n$ is the number of data points. In the latter, with the weights being real numbers, there is an infinite number of possible splits to pick from. This difference plays a key role in defining the kind of algorithms used to build each type of these decision trees.

For further reference:

OC1: A randomized algorithm for building oblique decision trees.

A System for Induction of Oblique Decision Trees

Inducing Oblique Decision Trees With Evolutionary Algorithms

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.