Suppose we use a decision tree to predict if a bank customer can pay back a credit. So it is a two class classification problem. Now we can make two mistakes:

  • $\alpha$ error: The customer can back the credit, but we predict he can't.
  • $\beta$ error: The customer can't pay back the credit, but we predict he can.

Now we know that $\beta$ errors are 123.4 times as expensive as $\alpha$ errors. But we only have a given set of data. In this set we have $n_1=10000$ customers who paid back the credit and $n_2 = 100$ customers who didn't.

How can the training of the decision tree be adjusted to account for the fact that $\beta$ errors are more expensive?

(Note: This is a theoretical question to learn about decision trees. I know about other classifiers like neural networks and I know of ensembles. However, I only want to know about decision trees here.)


2 Answers 2


Theoretically, decision tree algorithms specify the feature as well as the threshold that maximize the separation between classes at each node. This can be done by solving optimization problem related to the entropy at the children nodes. You can modify this optimization problem by including the miss- classification costs you have in the optimization problem so the algorithm will be biased toward the most expensive class

  • $\begingroup$ I don't really understand what you mean, could you be more specific? $\endgroup$
    – Thomas
    Jul 17, 2018 at 9:00

What you are looking for is "cost matrix". There are two options:

  1. Try to implement a cost matrix. (http://mlwiki.org/index.php/Cost_Matrix) It is easy.
  2. Second one is more complicated but effective one. Which is applying cost matrix to learning step of decision tree algorithms. Instead of using gini/entropy calculate your cost as -information gain in split level.

In many popular ml packages, as far as i know, there is no that type of cost learner algorithms. (sklearn, caret etc). I had to write myself, still didnt upload it to github.

However there is a beautiful work in the following github repo, have a look at it. It has ensemble implementation also: http://albahnsen.github.io/CostSensitiveClassification/


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.