What's the best way to use binned data in a tree-based model?

Question

I have some numeric data that has come 'binned', but the bins are not of equal sizes in terms of scale or quantile

For example, an age variable that is [0-16), [16-21), [21-30), [30-45), [45-65), [65, ]

If I leave it as a categorical variable, a tree will treat each category separately and discount the ordered relationship between the factors.

If I change it to an ordinal variable, e.g. [0, 1, 2, 3, 4, 5] and keep an array of labels for later reporting, the tree might split on, e.g. <2.5, which seems more natural to me, but then again the distance between 1 and 2 is not the same as the distance between 2 and 3 by any measure.

I'm leaning towards the second solution, but I would love some input!

Answer 1

For a complete decision tree, either of your proposed models would be able to represent the same set of concepts since a decision tree can be decomposed into a disjunction of conjunctions. I don't think the fact that the bins have different sizes is really an issue because placing any significance on the exact location of splits occurring "inside" a bin is probably not a good idea anyway. From a performance perspective, what probably matters most is how your data are distributed (i.e., does your concept split the data between a single pair of adjacent bins or is it more complicated).

Since you have the "random-forest" tag associated with your question, I'm inclined to agree with you that keeping them ordered is probably better because if you are generating many trees, the ordered representation will likely result in more terse/smaller trees.

Answer 2

Instead of using a random forest classifier, you could instead use a random forrest regression analysis. Use the mean value of the bin as the value (which will take into account the relative values as you suggest). This has the added benefit that if you get a dataset that has actual ages, you don't need to change the training, and you'll be used to translating the scores (ages) provided by the analytic to your bins.

Answer 3

You could also try to replace bins with Weight of Evidence values, and use those as your training set. See package woe if you are using R (smbinning or woeBinning are available as well, but don't catch non-linearities as well in my experience).