I frequently use Random Forest, Regularized Random Forest, Guided Random Forest, and similar tree models.
The size of the data that I'm dealing with has grown beyond what I can work around using HPC and parallelism. It's typically large due to row length (observations) not columns (features). The data is also often not normally distributed.
I have to make a choice between:
- Running a small number of trees (i.e. 50 or less) with either complete data or a relatively large and comparative sample
- Running several times the number of trees, but with a correspondingly scaled down sample size
There are work-arounds and for any 1 case -- for instance, I can do some ad hoc tests to see which I think will work better, but what I'm wondering is if there is a good theoretical (or robust empirical) reasoning to either guide the choice of approach over the other or to describe the tradeoff being made?
In other words, I'm hoping that someone more comfortable with the math, statistics, and theory underlying this (type of) algorithm can offer some generalizable insight.