1
$\begingroup$

Can anyone say the cross entropy Conceptually in machine learning for selecting best features, in simple way? I confused about that.

You do not really understand something unless you can explain it to your grandmother.

Albert Einstein

Improve this question
$\endgroup$
2
$\begingroup$

Entropy can be calculated using formula:-

$Entropy = -p\log_2 p − q\log_2 q$

Here $p$ and $q$ are probability of success and failure respectively in that node. Entropy is also used with categorical target variable. It chooses the split which has lowest entropy compared to parent node and other splits. The lesser the entropy, the better it is.

Steps to calculate entropy for a split:

  1. Calculate entropy of parent node
  2. Calculate entropy of each individual node of split and calculate weighted average of all sub-nodes available in split.
Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.