clustering plus linear model versus non linear (tree) model

Question

a team has to create models that predict the cost of deploying a machine over time. This is a regression problem.

The team is further divided into two groups, A and B.

Group A puts lots of attention on selecting attributes and finding rules that would segment the training set into clusters that are more or less homogeneous, and then uses a linear model to create predictions within the cluster.

Group B does not do clustering first, but instead includes the same attributes that team A uses for clustering into a non-linear model (let's say an ensemble of random forests or gradient boosting machine).

The results are similar (or slightly better using the non linear model).

How are results measured? mean squared error over a hold-out set.

The explanation appears to be that, by definition, tree models segment the population using attributes, so that the segments are as homogeneous or pure as possible, given the attributes.

So the work team A is doing to cluster the instances, the tree model is is also doing per se - because segmentation is embedded in tree models.

Does this explanation make sense?

Is it correct to infer that the approach of group B is less demanding in terms of time? (i.e. the model finds the attributes to segment the data as opposed to selecting the attributes manually)

Answer 1

With regards to the end of your question:

So the work team A is doing to cluster the instances, the tree model is is also doing per se - because segmentation is embedded in tree models. Does this explanation make sense?

Yes, I believe this is a reasonable summary. I wouldn't say the segmentation is "embedded" in the models but a necessary step in how these models operate, since they attempt to find points in the variables where we can create "pure clusters" after data follows the tree down to a given split.

Is it correct to infer that the approach of group B is less demanding in terms of time? (i.e. the model finds the attributes to segment the data as opposed to selecting the attributes manually)

I would imagine that relying on the tree implementation to derive your rules would be faster and less error prone than manual testing, yes.

Answer 2

The cluster variables might not be related to the response, and the number of observations in each cluster is smaller than in the full dataset.