Why do we need the entropy of parent node in the Information Gain. Information Gain = entropy(parent) - w * entropy(children)

We can compare the entropy of the children without the need for the parent entropy.


1 Answer 1


It's essential; you're computing gain from the parent to the same data split in the children! Not comparing children. A good split takes a high-entropy data set (lots of several different labels) and turns it into lower-entropy data sets (some labels on one child mostly, other labels on the other).

  • $\begingroup$ well. I can compute children entropy with only the parent data. So we don't need parent entropy to compare children entropy. For example if parent entopry is 0.5 and children entropies are 0.3 and 0.4 I can just compare -0.3 and -0.4. Why do fomula calculate information gain using parent entropy. $\endgroup$
    – Alireza HI
    Nov 25, 2018 at 15:38
  • $\begingroup$ Oh sure the parent entropy is constant when considering many splits. You are choosing the split with lowest child entropy regardless and that doesn't need parent entropy. However you may still need to compute information gain to, for example, see if the best split even achieves a positive gain at all. That does require knowing parent entropy. $\endgroup$
    – Sean Owen
    Nov 26, 2018 at 4:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.