For my machine learning task, I create a set of predictors. Predictors come in "bundles" - multi-dimensional measurements (3 or 4 - dimensional in my case).
The hole "bundle" makes sense only if it has been measured, and taken all together.
The problem is, different 'bundles' of predictors can be measured only for small part of the sample, and those parts don't necessary intersect for different 'bundles'.
As parts are small, imputing leads to considerable decrease in accuracy(catastrophical to be more accurate)
I could create dummy variables that would mark whether the measurement has taken place for each variable. The problem is, when random forests draws random variables, it does so individually.
So there are two basic ways to solve this problem: 1) Combine each "bundle" into one predictor. That is possible, but it seems information will be lost. 2) Make random forest draw variables not individually, but by obligatory "bundles".
Problem for random forest
As random forest draws variables randomly, it takes features that are useless (or much less useful) without other from their "bundle". I have a feeling that leads to a loss of accuracy.
For example I have variables
The problem is, variables
a_measure make sense only if variable
a is present, same for
b. So I either have to combine
a_measure into one variable, or make random forest draw both, in case at least one of them is drawn.
What are the best practice solutions for problems when different sets of predictors are measured for small parts of overall population, and these sets of predictors come in obligatory "bundles"?