What is the best approach to train a multi-category regression model (using XBoost decision trees ensemble)? What are the ups and downs of each one? For example, if I want to train a model to predict motor vehicle prices with a dataset containing trucks and motorbikes I can think of two options:

1) Training a general model with category-specific features like cargo-volume being different than zero for the category that it applies (trucks) and zero otherwise (motorbikes).

2) Training separate models, one for trucks and another for motorbikes, each one having its own set of specific features, but also features that are used in both like horse-power.

I can see that option (2) can more easily lead to overfitting, but I also have an intuition that it could perform better (even though I can't explain why).

Anyone can help me with this?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.