Building a linear regression model for every combination vs only one Machine Learning model

Question

So my question is more on the conceptual side.

Given a dataset, I want to predict a given continuous variable Y. Now, there are 3 features, 2 categorical and one numerical (integer only). I know that if I create a combination of the 2 categorical features I can use the numerical feature as an independent variable in the linear regression model to predict Y. For instance, for a combination of the 2 categorical features I build a linear model where the numerical feature is the independent variable. This yields good results because the relationship between Y and the numerical feature for a given combination will always be linear. However this also means that I might have to build 1000 linear regressions, one per combination. This of course sounds a bit strange to me, since I can use for instance a decision tree model without the need of creating the combinations of the categorical features. I'm trying to see the pros and cons of each methodology, but I'm having a hard time. Can anyone shed some light into this problem?

Example:

Imagine this dataset is from a massive bakery. The dataset is very large, over 100k instances. The categorical values are a machine ID and the recipe that it does. So the combinations are (machine, recipe). And the numerical feature is the number of times you do the recipe. Therefore I'm trying to predict the how long a given process will take. Now obviously, for every combination, the more you do the recipe, the more time it will take. The thing is, it seems super odd to just create a linear equation for every combination. Sure it works, but you end up with a enormous amount of linear equations which seems like it will take a lot more computational power than one single model.

Answer 1

In general the choice between a single model and multiple independent models depends on the expected level of dependency between the variables.

In your example, if two different machine ids implies two completely distinct machines with different characteristics, and therefore completely different relation between the recipe id and the duration, then it makes more sense to train a distinct model for each machine.

On the other hand if a lot of information is common between different machines and recipes, for example if the time for recipe R on machine M can be deduced from the time of recipe R on some other machines, then a single model could leverage this kind of correlation.

• An important factor to decide is the number of instances: the case of multiple independent models requires that you have a sample large enough (and representative enough) for every model.
• In case there are enough instances for every model, it's likely that the independent models would provide more accurate predictions since the model wouldn't conflate all the data.

At the end of the day, the only sure way to know which one is the best option is to run the experiments.