I built an R RandomForest Regression model. The source training data is a historical monthly report of all closed tickets, and the data for forecasting/prediction is a report of open tickets. These reports are generated by another team.
I test/train the model using two years of historical closed ticket data, and predict (forecast) a ŷ Completion Date for each open ticket.
The closed tickets training data looks like this:
ID | Dollars | Fruit | Etc | StartDate | CompletionDate |
---|---|---|---|---|---|
AA088 | 500 | Apple | ... | 1/1/2020 | 2/15/2020 |
AB100 | 1000 | Apple | ... | 1/1/2020 | 5/15/2020 |
AB101 | 2000 | Banana | ... | 1/1/2020 | 5/15/2020 |
BB723 | 5000 | Apple | ... | 1/5/2020 | 3/20/2020 |
BB724 | 3000 | Lime | ... | 1/5/2020 | 3/20/2020 |
BB725 | 1000 | Orange | ... | 1/5/2020 | 3/20/2020 |
The open ticket data looks similar, except it lacks CompletionDate, and sometimes various fields are "Unknown" at this time.
To build the model, I withhold "ID", make all categorical values factors, use CompletionDate as my y variable, and train the RandomForest on a majority of available features.
Recently, the team that generates this data threw a curve ball, rather than each row being a single record, rows are line-items of a higher level ticket! A majority of tickets have only one line-item, the remaining tickets can have between 2 and 6 line-items.
ID_Parent | ID_Row | Dollars | Fruit | Etc | StartDate | CompletionDate |
---|---|---|---|---|---|---|
AA | 088 | 500 | Apple | ... | 1/1/2020 | 2/15/2020 |
AB | 100 | 1000 | Apple | ... | 1/1/2020 | 5/15/2020 |
AB | 101 | 2000 | Banana | ... | 1/1/2020 | 5/15/2020 |
BB | 723 | 5000 | Apple | ... | 1/5/2020 | 3/20/2020 |
BB | 724 | 3000 | Lime | ... | 1/5/2020 | 3/20/2020 |
BB | 725 | 1000 | Orange | ... | 1/5/2020 | 3/20/2020 |
I have considered to summarize (rollup) records, which is easy for numeric value like Dollar (Sum(Dollars)
). I could concatenate the multiple categorical values, however, each factor is independent and has strong predictive value to the model (i.e. line items with "Apple" has a weight / meaning that would be lost if I simply concatenated as a string with the other row's value)
ID_Parent | SumDollars | ConcatenatedFruit | Etc | StartDate | CompletionDate |
---|---|---|---|---|---|
AA | 500 | Apple | ... | 1/1/2020 | 2/15/2020 |
AB | 3000 | Apple, Banana | ... | 1/1/2020 | 5/15/2020 |
BB | 9000 | Apple, Lime, Orange | ... | 1/5/2020 | 3/20/2020 |
How should I handle a categorical feature like Fruit that contains multiple factors?
Can RandomForest accept a feature that contains multiple factors? Do I need to use a different type of model?