1
$\begingroup$

So I just very recently learned about decision trees, and the different metrics for determining the best split when training the tree. I cannot seem to be able to find anything on which metric to use for certain situations, or which one is better to use for certain kinds of data?

The ones I'm comparing are:

  • Chi-Square
  • Gini Impurity
  • Gini Index (Gini Coefficient)
  • Variance
  • Information Gain
  • Information Gain Ratio

When should I use which ones, or is there an insignificant difference between all?

$\endgroup$
1
$\begingroup$

This is not really a question about decision trees, but about properties to difference metrics. I would be surprised if you find significant differences in the final performance of your decision tree, for different metrics, but if you do find big differences, please comment!

So I don't think it matters very much what you choose here, but for example, the chi-squared numerical value has a specific meaning that is only valid if the data is normally distributed. If it is not, then the chi-squared can be artificially big, or small. But the best split will probably be the one with the lowest chi-squared, so that's why I don't think there is a big impact, even if your data is not normally distributed.

About the Gini coefficient: A given split results in two samples that you're comparing with the truth. The Gini coefficient has a downward bias for small samples, so if one of your samples after splitting is small, then your metric will be biased. That's not great, so probably, don't use the Gini coefficient in a decision tree.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.