I am interested in finding out how decision trees chose the order in which they split. I understand that splitting is based in information gain. The attribute with the lowest information gain is chosen as the root node.
If I had a data set with columns:
- credit standing
- marriage status
and I was interested in finding out what determines a person to have a good or bad credit status, Am I correct in saying, I calculate entropy and information gain for each of these categorical attributes against the independent attribute i.e what I am investigating (credit standing), and that the calculation with the lowest information gain is chosen as the root node.
For example, if this root node (first split) was Age, Is the entropy and information gain of Age (the new independent attribute) against the remaining attributes (marriage status and income) calculated, and the calculation with the lowest information gain is then chosen as the second split node, and so on?
credit standing vs age = 0.01 credit standing vs status = 0.1 credit standing vs income = 0.2
Age is chosen as root (first split) node.
then, information gain:
age vs status = 0.2 age vs income = 0.1
income is chosen as second split node.
Am I understanding this correctly?