# Decision tree ordering

I am interested in finding out how decision trees chose the order in which they split. I understand that splitting is based in information gain. The attribute with the lowest information gain is chosen as the root node.

If I had a data set with columns:

1. credit standing
2. age
3. income,
4. marriage status

and I was interested in finding out what determines a person to have a good or bad credit status, Am I correct in saying, I calculate entropy and information gain for each of these categorical attributes against the independent attribute i.e what I am investigating (credit standing), and that the calculation with the lowest information gain is chosen as the root node.

For example, if this root node (first split) was Age, Is the entropy and information gain of Age (the new independent attribute) against the remaining attributes (marriage status and income) calculated, and the calculation with the lowest information gain is then chosen as the second split node, and so on?

ie.

information gain:

credit standing vs age    = 0.01
credit standing vs status = 0.1
credit standing vs income = 0.2


Age is chosen as root (first split) node.

then, information gain:

age vs status = 0.2
age vs income = 0.1


income is chosen as second split node.

Am I understanding this correctly?

• You partition by the attribute offering the highest, not lowest, information gain (entropy reduction). – Emre Nov 17 '17 at 0:27

Here in your Scenario you need to select the one with more Information Gain rather than the least one and the process goes on till you reach the last feature/last node.

Go through this Link. I think your doing it vice versa, I agree with Emre.

In the above link it was explained with an example to decide whether to play tennis or not.

Do let me know if you have any issues.

• Link only answers are frowned upon for a variety of reasons. One big one reason is that links rot. Please spend a couple of minutes and at least summarize the story that the link tells. – Stephen Rauch Nov 17 '17 at 3:57
• I understand. The reason why I appended the link is, Information Gain and Entropy was explained with an example. I thought it would be good if she can go through that example for better understanding. – Toros91 Nov 17 '17 at 4:00