Given either, different decision trees each based on a particular feature value (like separate models for each male and female) or a single decision tree, should both give the same result?

  • $\begingroup$ So do you have just one feature which is gender and you need to predict the output of some response variable? $\endgroup$ Dec 31 '17 at 12:35
  • $\begingroup$ No I am making different models based on of the feature values (gender in this case) all and other features are same for all the models $\endgroup$
    – tam
    Dec 31 '17 at 17:09
  • $\begingroup$ Should depend on how you combine the results of individual trees. What is your strategy for combining them? You know if you train your individual trees on a subset of features and combine them you have a Random Forest which results differently from an individual decision tree. That's why it's important to know how you combine individual trees I suppose. $\endgroup$ Jan 2 '18 at 8:19

Decision tree are deterministic so will always make the same split if given the same data.

A single decision tree will be make splits conditional on previous splits (greedily taking the best split for either the previous split feature or other features). Separate trees per feature will only split conditional on the feature the tree has access to.

In general, most problems are best solved by a single decision tree. Decision trees will automatically find the best features and best split points.


Decision tree will split your data based on the most relevant features, it is not neccessary to give each decision tree with different feature.

Let say an example, You have 3 features named as gender,profession and fare and the output required is some column. The most relevent features can be captured by different approaches, here i taking entropy and information gain as example.

Lets say information gained by gender is more, then decision tree will split to two nodes, Male and Female where each node will have some samples splited to each based on the information gain calculated against output column. The information gain and entropy of splited data calculated again to find the next node, the resultant node will be any one of the feature having most information gain. The process will continue untill you reach the maximum depth.

The disadvantages of decision tree is overfitting (depth of decision tree will be higher) and high variance, to overcome this we often use early stopping or random forest where the data trained by multiple tree (Where each tree can have shuffled training data) and predict based on the majority votes given by each tree

Example image with depth of 4

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.