Are entropy and cross-entropy the same thing as per the basic definition?

If there is a difference:

Decision tree splits take on entropy or Gini index, can we use cross-entropy to split decision trees? OR should I use it as an evaluation metric after running the decision tree algorithm?

Also, does the decision tree algorithm assumes any distribution?

If yes,

then how can we use the KL Divergence metric?

I am just trying to link a few concepts in a broader view.

These are my concerns with respect to the multiclass decision tree.


1 Answer 1


Entropy and cross-entropy are different concepts. Entropy quantifies the uncertainty in a single random variable. Cross-entropy quantifies the uncertainty between two distributions.

The decision tree algorithm does not make any assumptions about the distribution of the features.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.