Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
So I just very recently learned about decision trees, and the different metrics for determining the best split when training the tree. I cannot seem to be able to find anything on which metric to use for certain situations, or which one is better to use for certain kinds of data?
The ones I'm comparing are:
When should I use which ones, or is there an insignificant difference between all?
This is not really a question about decision trees, but about properties to difference metrics. I would be surprised if you find significant differences in the final performance of your decision tree, for different metrics, but if you do find big differences, please comment!
So I don't think it matters very much what you choose here, but for example, the chi-squared numerical value has a specific meaning that is only valid if the data is normally distributed. If it is not, then the chi-squared can be artificially big, or small. But the best split will probably be the one with the lowest chi-squared, so that's why I don't think there is a big impact, even if your data is not normally distributed.
About the Gini coefficient: A given split results in two samples that you're comparing with the truth. The Gini coefficient has a downward bias for small samples, so if one of your samples after splitting is small, then your metric will be biased. That's not great, so probably, don't use the Gini coefficient in a decision tree.
