Decision Tree Optimize Deviation From Objective

Question

I have the following problem: I have three classes/modes, let's call them car, bike, and walking. For any given test data instance with some environmental variables such as distance, road quality etc, I would like to predict the cheapest mode. In all instances in the training and test data set, each of the modes is associated with a cost. I would like to use a decision tree due to it being easy-to-understand. Is it possible to change the "performance metric" during training such that the final tree minimizes the cost deviation due to wrongful classification rather than maximizing the accuracy?

I know that the goal of minimizing the cost deviation can be achieved by employing three (linear) regression models, and then choosing the cheapest mode, but I would like to keep the "easy-to-understand" property of decision trees. Also, some of the explaining variables are non-linear (for example, walking is preferred, if it is neither too hot nor too cold).

Is this possible in general?

Answer 1

It is possible, is not an easy task but it's possible.

Custom Loss Function

Gradient boosting is widely used in industry and has won many Kaggle competitions. The internet already has many good explanations of gradient boosting (we’ve even shared some selected links in the references), but we’ve noticed a lack of information about custom loss functions: the why, when, and how. This post is our attempt to summarize the importance of custom loss functions in many real-world problems — and how to implement them with the LightGBM gradient boosting package.