Decision tree with multiple outputs

Question

I have a sample with 10 independent variables (X1, X2, X3 ....), and multiple output labels (y1, y2, y3).

Here y1 will depend on X1, X2

y2 will depend on X3, X4 and so on.

y1, y2, y3 might or might not be correlated.

Can you please suggest pros and cons of clubbing this into single model, or should i go with multiple models with single output

X1,X2,y1

X3,X4,y2

.....

Thanks

Answer 1

You basically answered your question: if y1, y2, y3 are independent, there is no point to use a single model. If they are dependent (say, y1=1 implies that y2 is not 2), then the single model helps to exclude the non-existent case.

However, if y1 is a function of x1 and x2, and y2 is a function of x3 and x4, and x1,x2,x3, and x4 are independent, y1 and y2 are also independent.

I would start from whatever seems simpler and more logical. Based on your description of weak dependence between variables, multiple models seem more promising to me.

Answer 2

Though I believe DT/RF will not have any challenge in a single model but I will prefer the other choice.

Multiple models

Pros -
1. Simple and interpretable modelling
2. You have lesser number of dimension per data count
3. Simple maintenance and troubleshooting. Every new developer will not have to waste some time in figuring out this point.
4. If they are independent and only correlated to respective Y. This can have a relation to the data domain. So keeping separate is better for future data decisions
5. Liberty to use different models for the different dataset as per the under-lying data pattern
6. Inherent loose-coupling and low cohesion

Cons
1. Manage 3 code/production instance