Let's say I have an expertise in the domain knowledge of the dataset I am working on. I know that part of my dataset acts 100% differently than the rest.

Also, it is straightforward to check if a row belongs to one part of the dataset or the other with a couple of simple if-else.

Should I split my dataset upfront and create two different prediction models for the two parts of the dataset? Or should I keep one model and try to improve it?

Then when I want to use the model for predictions, I check the input values and see in which category it belongs and I call the appropriate model for it.

Yes, you can divide your dataset and use different prediction models for different part of your dataset. Take advantage of your domain knowledge to build a good model.

After all, you can actually count the following as a single model $$f(x) = \begin{cases} f_1(x) & ,x \in A \\ f_2(x) & ,x \in A^c \end{cases}$$

As @Siong They Goh wrote, since it is straightforward to check to which part the model belongs, the classification algorithm might find it in the combined set. If the partitioning will be very strong, this split will be important in the model (e.g., the root in the decision tree) and afterwards the classifier will have to learn the proper model for each part. That will happen in a scenario in which you have unrelated parts. Note that in such situation it won't matter much whether you will split the data set or not. The benefit that you will gain will be from preventing noise to mislead your classifier (e.g. an unrelated feature that by random looks informative).

A more interesting scenario will be when the division into parts is informative but there are other features that are even more informative. In this case given a joint data set, it will be easier for the classifier to use them. For example, if your data set is small, it will be less representative and the influence of noise will be stronger, two effect that will become stronger when splitting it.

I suggest that you will choose a small benchmark classifier (e.g., a size bounded decision tree) and check its performance on both variants. This way you will be able to know which solution fits you more.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.