I want to use some Decision Tree learning, such as the Random Forest classifier.

I have data of different types: continuous, discrete and categorical. How do I have to preprocess data in order to have consistent results?


One of the benefits of decision trees is that ordinal (continuous or discrete) input data does not require any significant preprocessing. In fact, the results should be consistent regardless of any scaling or translational normalization, since the trees can choose equivalent splitting points. The best preprocessing for decision trees is typically whatever is easiest or whatever is best for visualization, as long as it doesn't change the relative order of values within each data dimension.

Categorical inputs, which have no sensible order, are a special case. If your random forest implementation doesn't have a built-in way to deal with categorical input, you should probably use a 1-hot encoding:

  • If a categorical value has $n$ categories, you encode the value using $n$ dimensions, one corresponding to each category.
  • For each data point, if it is in category $k$, the corresponding $k$th dimension is set to 1, while the rest are set to 0.

This 1-hot encoding allows decision trees to perform category equality tests in one split since inequality splits on non-ordinal data doesn't make much sense.

  • $\begingroup$ can random forest handle continuous data? $\endgroup$ – Mubeen Khan Oct 1 '19 at 15:25
  • $\begingroup$ @MubeenKhan Yes. Conveniently, if you have N training data points, the algorithm only has to consider N values, even if the data is continuous. The inequalities it finds during training (should) generalize for continuous data. $\endgroup$ – Joshua Little Oct 3 '19 at 14:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.