0
$\begingroup$

usually a decision tree has one root node, some nodes, and some leaves.

lots tutorial illustrate this as something like binary tree.

is it possible more than 2 nodes away from a node in a decision tree?

enter image description here

this image comes from this post

by "more than 2 nodes", i mean there are more than 3 splits (in this case, 3, Low, Med, High) away from the root node.

if it is reasonable in real life application, plz provide an open dataset on which a decision tree would spit more than 2 nodes, and a piece of sklearn code.

$\endgroup$
  • $\begingroup$ what do you mean by "more 2 leaves away from a node" ? $\endgroup$ – vb_rises May 22 at 2:09
  • $\begingroup$ A binary tree has to have 2 nodes. Other trees can have more than 2 nodes $\endgroup$ – Sandeep B May 22 at 9:16
  • $\begingroup$ stats.stackexchange.com/q/12187/232706 $\endgroup$ – Ben Reiniger Oct 24 at 13:38
1
$\begingroup$

It is possible to make more than a binary split in a decision tree. Chi-square automatic interaction detection (CHAID) is an algorithm for doing more than binary splits.

However, scikit-learn only supports binary splits for many reasons. One primary reason to limit to just binary splits is that the library can support as many splitting criteria as possible with the same API. For example, Gini Impurity only supports binary splits.

In practice, only supporting binary splits is not an issue because a series of binary splits can model any number of simultaneous splits.

$\endgroup$
0
$\begingroup$

I think that scikit-learn only implements binary trees. However, you can turn your example into a binary tree so you can use scikit-learn:

Savings == Low
    True => Assets == Low
        True => Bad Risk
        False => Good Risk
    False => Savings == Med
        True => Good Credit Risk
        False => Income <= 30K
            True => Bad Risk
            False => Good Risk
$\endgroup$
0
$\begingroup$

IMO decision trees are - by definition - designed so that the single "best" split is chosen in each step (Introduction to Statistical Learning, Ch. 8.1). I think you need to split on values low, medium, high, in which case a first split would e.g. occur at (low) vs. (medium, high) and a later split would be between (medium, high), whatever gives the best fit.

Single decision trees often do not have a very good predictive capacity (see. Introduction to Statistical Learning, Ch. 8.2). If you are interested in accuracy of prediction, you should go a step further and grow a random forest with "bagging" or even better "boosting" on many trees (or "ensambles of trees"). In this case, many trees are grown, and they all together make a "vote" on how to predict some outcome.

Random Forest in scikit-learn: https://scikit-learn.org/stable/modules/ensemble.html

Prominent boosting methods are e.g. catboost (https://catboost.ai/docs/concepts/about.html) or lightGBM (https://lightgbm.readthedocs.io/en/latest/).

Alternative (non-tree based) models will be able to make a differentiation by three classes without any problem (ISL, Ch. 4). One example is a logistic model (or "logit") of form risk = b0 + b1*savings. In this models you can also calculate marginal effects, telling you, in case someone moves from class A to class B, by how much will the probability of being a "bad risk" change (marginal effects). https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html

You can find a bunch of good code directly covering topics mentionned here and discussed in the book "Introduction to Statistical Learning" online: https://github.com/JWarmenhoven/ISLR-python

In summary: If you are interested in prediction, don't stick to simple decision trees, but move on to something else.

$\endgroup$
  • $\begingroup$ what does IMO mean, dose IMO mean "International Maritime Organization"? it seems not. $\endgroup$ – shi95 May 22 at 20:48
  • $\begingroup$ in my opinion... has nothing, whatsoever, to do with ships $\endgroup$ – Peter May 22 at 20:50
0
$\begingroup$

if it is reasonable in real life application

From my understanding of the question, Technically, I think it is perfectly reasonable to have a decision tree/forest splitting into 3 or more nodes from the root node. Kindly check this example if you not. In this example, the predictor variable is whether to play tennis on a given day or not depending on how the climate is on that day.

enter image description here

But I'm not sure whether this is practically possible given the tree structure and the splitting strategy involves find the nodes which correspond to the most reduction in entropy in the model or highest gain in information gain.

P.S: If this answer doesn't give you any new information or clarity, please say so. I'll remove it in order not to misguide or anyone.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.