Can anyone say the cross entropy Conceptually in machine learning for selecting best features, in simple way? I confused about that.
You do not really understand something unless you can explain it to your grandmother.
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
Entropy can be calculated using formula:-
$Entropy = -p\log_2 p − q\log_2 q$
Here $p$ and $q$ are probability of success and failure respectively in that node. Entropy is also used with categorical target variable. It chooses the split which has lowest entropy compared to parent node and other splits. The lesser the entropy, the better it is.
Steps to calculate entropy for a split: