I have read a lot of articles about decision trees, and every one of them only focused on telling how a feature/column is considered for split, based on criterion like gini index, entropy, chi-square and information gain. But, not one talked about the observation part.
Example: Let's say I have a dataset with 3 independent features and 1 discrete target variable, namely
height_in_cm(like 130, 140),
performance_in_class(like below average, average, very good),
class(like 7th, 8th or 10th class) and
plays_cricket(1 for yes & 0 for no) is the target variable. So, with target variable as the root node and then for splits, I may try all the features iteratively and settle with the one that I had most information gain with or with most pure nodes. ex, let's start with any variable, let's take the first one,
height_in_metres and after two splits with two child nodes, like height < 120 in one child node and another child node with height > 120 and i will then calculate gini impurity and they turned out to be 0.45 and 0.49 respectively.
- just like i iteratively try all the feature combinations, do i also need to try all the combinations of a feature split, for above case, height < 100 & height > 100, then height < & height > 110 & height < 90 & height > 90 and so on. How to do this and what's the efficient way.
- just like there are metrics like gini impurity and entropy to measure the quality of a feature split, are there any metrics to measure the quality of split based on observations?