I can’t really understand, why my decision tree only splits to the left.
I originally have 2 categorical features (further named feature 0 and 1), which I concat to one feature since feature 1 is dependent on feature 0.
Feature 0: A Feature 1: b
New Feature: A_b
Now I use this new feature (One-Hot-Encoded) to predict some categories.
There is just the problem that, by doing that the decision tree only grows to the left. Although the Gini on the right leaf nodes are still really high, and there are also enough samples to split further on the right subsets.
But if I just use the 2 features without concatenation it splits in both direction. But I can’t do that since both features are dependent on each other.
By dependent I mean the following:
Not every category of feature 1 appears with feature 0. Hence the decision tree would build rules, which aren’t representing what could be possible in reality.