1
$\begingroup$

enter image description here

For the above table, midpoints for possible split points are 22.5 and 35. I have calculated the entropy and gain for each value and 35 had the minimum Entropy and highest gain. Is it correct ?

Given High -> (-), and Low -> (+)

D<22.5 => [0+, 2-], Entropy (D<22.5) = 0, since all the values are of the same class High.

D>22.5 => [2+, 2-], Entropy (D>22.5) = 1, since the values are distributed equally among Low and High classes.

D<35 => [2+, 3-], Entropy (D<35) = -[2/6 x $log_2$(2/6) + 3/6 x $log_2$⁡(3/6)]= 0.5

D>35 => [0+, 1-], Entropy (D>35) = 0, since all the values are of the same class High

Gain (D, Age>22.5) = 0.918 - 2/6 (0) - 4/6 (1) = 0.2513

Gain (D, Age>35) = 0.918 - 5/6 (0.5) - 1/6 (0) = 0.5103

Is that right?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

For 35 split, there is an error in the denominator. It should be 5 as the total number of items for D<35 is 5

D<35 => [2+, 3-], Entropy (D<35) = -[2/5 x $log_2$(2/5) + 3/5 x $log_2$⁡(3/5)]

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you jdsuryap , this was helpful. Is correct to say that though the Entropy of D<22.5 is higher than Entropy of D<35; however the gain for D<22.5 is higher. $\endgroup$
    – Ali
    Apr 13, 2021 at 15:49
  • $\begingroup$ Perhaps not @Ali. If you would try fixing the error as I proposed, the correct entropy of 35 split is higher. Hence, the information gain for split 22.5 is higher. $\endgroup$
    – jdsurya
    Apr 13, 2021 at 17:56
  • $\begingroup$ Thank you. I did already. Entropy (D<35) = -[2/5 x 〖log_2 (〗⁡〖2/5)〗+3/5 x 〖log_2 (〗⁡〖3/5)〗 ]= 0.9709 D<35  [2+, 3-] . So the Entropy of 22.5 is 1 and Entropy of 35 is 09709 however the gain for 22.5 will be higher though its entropy is lower. $\endgroup$
    – Ali
    Apr 15, 2021 at 6:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.