How do we pre-process data for very sparse features for a decision tree?
From this Turi documentation for decision trees
It mentioned this:
Why chose decision trees? Different kinds of models have different advantages. The decision tree model is very good at handling tabular data with numerical features, or categorical features with fewer than hundreds of categories. Unlike linear models, decision trees are able to capture non-linear interaction between the features and the target.
One important note is that tree based models are not designed to work with very sparse features. When dealing with sparse input data (e.g. categorical features with large dimension), we can either pre-process the sparse features to generate numerical statistics, or switch to a linear model, which is better suited for such scenarios.
If we have a categorical column that is very sparse (many 0's), then for that category what type of numerical statistics can we transform for each row?