How are weights considered in a decision tree when we want to maximize Information Gain? In other words, what would the entropy calculation become when weights are involved?

I can guess either $$ e_1 = \sum_{i} w_iP(X=i)log(P(X=i))$$ or $$ e_2 = \sum_{i} w_iP(X=i)log(w_iP(X=i))$$ should be used as the new weighted entropy but I'm not sure. And is the rest of the algorithm same as before, using $e_1$ or $e_2$ instead of normal entropy, or are there other changes as well?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.