Let's say I have a dataset consisting of 100 features and a binary target variable. On exploring the data, I see that Feature 10 which is binary, seems to split the data in an 80:20 ratio with respect to the target variable. Does it then make sense to partition the data into "Feature 10 is 0" and "Feature 10 is 1" and try to build separate classifiers for both cases? Or is this what models like Random Forests do under the hood?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ You might have business constraints for which it would make sense to split the model. $\endgroup$– Lucas MorinCommented Jul 4, 2022 at 7:24
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
It can make sense, but more commonly I think it doesn't.
When you split into two models, those two models can't help each other: any common trends between your sub-populations will have to be modeled by both models, and so probably won't be estimated as well as by one larger model. But, if those populations are really different, that can be a benefit rather than a detriment, especially for low-capacity models. As for tree models, note that splitting into two populations would be exactly equivalent to splitting on that variable as the first split in every tree (extremely unlikely, especially if you're sampling columns).
answered Jan 29, 2020 at 13:14
$\begingroup$
$\endgroup$
Yes, there is no need for you to manually split the dataset into two parts and train separate classifiers for them.
That's something that decision trees (eg. Random Forest), will do, as you say, under the hood.
The manipulation that you could do on your dataset is probably the dimensionality reduction - but, this is also done, informally, by decision trees. See "Random Forests / Ensemble Trees" section here.
