Skip to main content
deleted 4 characters in body
Source Link

how are split decisions for observations(not features) made in decision trees

I have read a lot of articles about decision trees, and every one of them only focused on telling how a feature/column is considered for split, based on criterion like gini index, entropy, chi-square and information gain. But, not one talked about the observation part.

Example: Let's say I have a dataset with 3 independent features and 1 discrete target variable, namely height_in_metresheight_in_cm(like 130, 140), performance_in_class(like below average, average, very good), class(like 7ht7th, 8th or 10th class) and plays_cricket(1 for yes & 0 for no) is the target variable. So, with target variable as the root node and then for splits, I may try all with all the featurefeatures iteratively and settle with the one that I had most information gain with or with most pure nodes. ex, let's start with any variable, let's take the first one, height_in_metres and after two splits with two child nodes, like height < 120 in one child node and another child node with height > 120 and i will then calculate gini impurity and they turned out to be 0.45 and 0.49 respectively.

Questions:

  1. just like i iteratively try all the feature combinations, do i also need to try all the combinations of a feature split, for above case, height < 100 & height > 100, then height < & height > 110 & height < 90 & height > 90 and so on. How to do this and what's the efficient way.
  2. just like there are metrics like gini impurity and entropy to measure the quality of a feature split, are there any metrics to measure the quality of split based on observations?

how are split decisions for observations made in decision trees

I have read a lot of articles about decision trees, and every one of them only focused on telling how a feature/column is considered for split, based on criterion like gini index, entropy, chi-square and information gain. But, not one talked about the observation part.

Example: Let's say I have a dataset with 3 independent features and 1 discrete target variable, namely height_in_metres(like 130, 140), performance_in_class(like below average, average, very good), class(like 7ht, 8th or 10th class) and plays_cricket(1 for yes & 0 for no) is the target variable. So, with target variable as the root node and then for splits, I may try all with all the feature iteratively and settle with the one that I had most information gain with or with most pure nodes. ex, let's start with any variable, let's take the first one, height_in_metres and after two splits with two child nodes, like height < 120 in one child node and another child node with height > 120 and i will then calculate gini impurity and they turned out to be 0.45 and 0.49 respectively.

Questions:

  1. just like i iteratively try all the feature combinations, do i also need to try all the combinations of a feature split, for above case, height < 100 & height > 100, then height < & height > 110 & height < 90 & height > 90 and so on. How to do this and what's the efficient way.
  2. just like there are metrics like gini impurity and entropy to measure the quality of a feature split, are there any metrics to measure the quality of split based on observations?

how are split decisions for observations(not features) made in decision trees

I have read a lot of articles about decision trees, and every one of them only focused on telling how a feature/column is considered for split, based on criterion like gini index, entropy, chi-square and information gain. But, not one talked about the observation part.

Example: Let's say I have a dataset with 3 independent features and 1 discrete target variable, namely height_in_cm(like 130, 140), performance_in_class(like below average, average, very good), class(like 7th, 8th or 10th class) and plays_cricket(1 for yes & 0 for no) is the target variable. So, with target variable as the root node and then for splits, I may try all the features iteratively and settle with the one that I had most information gain with or with most pure nodes. ex, let's start with any variable, let's take the first one, height_in_metres and after two splits with two child nodes, like height < 120 in one child node and another child node with height > 120 and i will then calculate gini impurity and they turned out to be 0.45 and 0.49 respectively.

Questions:

  1. just like i iteratively try all the feature combinations, do i also need to try all the combinations of a feature split, for above case, height < 100 & height > 100, then height < & height > 110 & height < 90 & height > 90 and so on. How to do this and what's the efficient way.
  2. just like there are metrics like gini impurity and entropy to measure the quality of a feature split, are there any metrics to measure the quality of split based on observations?
Source Link

how are split decisions for observations made in decision trees

I have read a lot of articles about decision trees, and every one of them only focused on telling how a feature/column is considered for split, based on criterion like gini index, entropy, chi-square and information gain. But, not one talked about the observation part.

Example: Let's say I have a dataset with 3 independent features and 1 discrete target variable, namely height_in_metres(like 130, 140), performance_in_class(like below average, average, very good), class(like 7ht, 8th or 10th class) and plays_cricket(1 for yes & 0 for no) is the target variable. So, with target variable as the root node and then for splits, I may try all with all the feature iteratively and settle with the one that I had most information gain with or with most pure nodes. ex, let's start with any variable, let's take the first one, height_in_metres and after two splits with two child nodes, like height < 120 in one child node and another child node with height > 120 and i will then calculate gini impurity and they turned out to be 0.45 and 0.49 respectively.

Questions:

  1. just like i iteratively try all the feature combinations, do i also need to try all the combinations of a feature split, for above case, height < 100 & height > 100, then height < & height > 110 & height < 90 & height > 90 and so on. How to do this and what's the efficient way.
  2. just like there are metrics like gini impurity and entropy to measure the quality of a feature split, are there any metrics to measure the quality of split based on observations?