Given: a data frame of 2.5 million records and 25 columns, which contains 1 grouping column with a variable number of groups (e.g. 2.5M records / 10 groups today; 2.5M records / 50 groups tomorrow).
How can I build a decision tree for each group?
I was going to do this using Python but thought there might be an easier way in R (tapply?). Here was my initial thought process in Python:
- instantiate an empty dictionary to hold the prediction results
- identify the unique ids in the grouping column and iterate over the rows of data in a
for
loop - in each loop, build the decision tree, predict and store the results in the dictionary as a
{grouping_var:prediction}
pair
Thought on how to do this faster or more efficiently in R?