I have aproblem where I need to predict when a Truck arrives to pickup something.

Say we have formulated that a binary classification model, where

0: The truck coming for pickup today

1:The truck comes for pickup sometime in the future after today.

Next, it was decided to expand the 2 class problem to a three class one, as follows:

0: The truck coming for pickup today,

1: The truck comes for pickup tomorrow

3: The Truck comes for pickup sometime in the future after tomorrow.

So, the 0-class remains as it is. A Random Forest (ranger in R) model was applied in both situations and the results compared. SInce the 0-class remains as is, we would expect the number of 0-predictions to be close from both models. But that is not the case: there is about a 30% difference in the number of 0-class predictions in the two models, which I find unintuitive. Is this behavior not really as unintuitive or is somethign wrong?

  • $\begingroup$ How about the accuracy or whatever metric you have chosen for the first class? $\endgroup$
    – The Lyrist
    Jun 19 '18 at 11:24

Trucks could come any of the days in the future, and one could represent this as a histogram. Since there is an infinite days in the future, the histogram with N bins cuts off the distribution and the last bin represents after-N days. With a 2-bin histogram, a lot of samples will be in that second bin. Some of these are quite close to the decision border between classes, some are really far away. Some were in class 0 but just barely, because of the large distance to samples far away in class 1. When introducing another class/bin, the samples that were close the boundary from both sides will likely move into the new bin. Note that the decision boundaries for all classes will shift!

Your classes represent a discretization of a continuous variable (time that the truck arrive). You might want to increase the number of classes to more closely represent the underlying distribution. For instance by going to 2/4/6 hour periods, or increasing the number of days. Or having a continuous output.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.