I have been reading some stackoverflow questions on how to handle nominal features for decision tree (sklearn implementation).
One of the answer states that :
Using a OneHotEncoder is the only current valid way, allowing arbitrary splits not dependent on the label ordering, but is computationally expensive.
But if we use LabelEncoder and make our tree go deep enough it will eventually isolate each category and the output remains the same.
So whats is the advantage of OneHotEncoding then?
I would not say that using One-Hot-Encoder to handle nominal features is an optimal way. Even it might affect adversely your Decision Tree model in terms of performance.
Theoretically, a Decision Tree does not need a one-hot-encoding for categorical features, besides it will increase the number of computations and will result in a relatively inefficient model. Unlike non-tree algorithms (e.g. regression algorithms), a Decision Tree cannot be biased by the label encoding of categorical features because it uses splits to reach the optimal result.
Experimental results show that one-hot-encoding gives a rise to sparsity in the trees which might drastically drop the performance of the model. Here is a wonderful analysis and explanation of the effect of one-hot-encoding on Tree-based algorithms. Also, check this useful source.
