I have a dataset containing several features, which have class values (e.g. DBF4, JUL23, ...). In a classification problem and when using decision trees, is it better to convert these values as new binary features: so DBF4 will become a feature, which value is either 0 or 1, or is it better to keep them as they are. Knowing that there are a lot of values (more than the number or rows actually). If this is the case, is there a reason why is it better ?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
In general, it’s better NOT to binarize them if the decision tree algorithm that you are using supports categorical features, as variables with fewer levels are less likely to be used in splits. Note however that not all implementations in all software are able to use categorical features without one-hot encoding – for example, most R implementations of decision trees and tree ensembles support categorical features natively, whereas scikit-learn’s and spark’s do not.
Here’s a blog post with a comparison of categorical as-is vs. one-hot encoded for random forests: https://roamanalytics.com/2016/10/28/are-categorical-variables-getting-lost-in-your-random-forests/
