0
$\begingroup$

How are weights considered in a decision tree when we want to maximize Information Gain? In other words, what would the entropy calculation become when weights are involved?

I can guess either $$ e_1 = \sum_{i} w_iP(X=i)log(P(X=i))$$ or $$ e_2 = \sum_{i} w_iP(X=i)log(w_iP(X=i))$$ should be used as the new weighted entropy but I'm not sure. And is the rest of the algorithm same as before, using $e_1$ or $e_2$ instead of normal entropy, or are there other changes as well?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.