4
$\begingroup$

In scikit-learn documentation and in decision tree learning Wikipedia article there is mention of "There are concepts that are hard to learn because decision trees do not express them easily, such as XOR, parity or multiplexer problems."

I can't recall knowing such problem types.

What are examples (and possibly related datasets) of such problems?

$\endgroup$

1 Answer 1

5
$\begingroup$

Below is an example of XOR dataset for classification. As you can see, decision trees perform pretty poorly on this dataset. Reason is decision trees splits space into rectangular regions. Therefore they are not pretty good with this kind of distributions.

If you really want to use trees in that sort of situation, it is interesting to use so-called rotation trees. Rotation is about performing PCA (principal components analysis) on input features while learning trees. Using it, decision trees can then build non-rectangular regions.

XOR machine learning

Also, here is a playground to test gradient boosting algorithms including on XOR dataset. It is really interesting ! You can click on "rotate trees" to activate rotation. Gradient boosting playground

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.