2022 Developer Survey is open! Take survey.

# Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

691 questions
Filter by
Sorted by
Tagged with
15 views

### Do cost-complexity pruned trees perform better relative to unpruned trees?

I came across an adapted question from the famous ISLR book and realise I am unsure of the answer. Does anyone know? Interested in the intuition here! Cost-complexity pruned trees with $\alpha=1$ ...
1 vote
23 views

### How do I design a random forest split with a "not sure" category?

Let's say I have data with two target labels, A and B. I want to design a random forest that has three outputs: A, B and Not sure. Items in the Not sure category would be a mix of A and B that would ...
• 249
19 views

### How does ExtraTrees (Extremely Randomized Trees) learn?

I'm trying to understand the difference between random forests and extremely randomized trees (https://orbi.uliege.be/bitstream/2268/9357/1/geurts-mlj-advance.pdf) I understand that extratrees uses ...
• 249
1 vote
11 views

### Machine learning frameworks for tree-based models

Background: Its well known that Pytorch and TensorFlow are currently the most used frameworks for Deep Learning (DL) research. As far as I know, most researchers (applied or theoretical) that ...
• 89
10 views

### forcing decision tree use specific features first

My goal it to force some feature used firstly to split tree. Below, the function splitted tree using feature_3 first. For instance, is there a way to force to use feature_2 first instead of feature_3 ?...
• 105
15 views

### Information gain calculation for decision tree

I am having trouble figuring out how to calculate information gain in decision trees. Suppose a decision tree node splits the data of 5 red and 5 black balls into 2 branches: 4 reds in the right child ...
46 views

### What is the max number of leaf nodes in a classifcation decision tree?

Let's assume that we have n observations and p predictors and we have in a n>>p situation. All predictors are binary. What is the max number of leaf nodes that we can have in the tree? and what ...
20 views

### Decision tree from boosted tree regressor Google bigquery ML

If I set the num_parallel tree to 1 and max_iteration to 1 in boosted_tree_regressor of Google Big Query ML will it work as Decision tree regressor ? Also can such decision tree give negative ...
26 views

### Identifying subsets of values significant to the total sum

Imagine a set of products in a store, with all the different attributes assigned to them - some of these hierarchical (e.g. categories), and some not (e.g. brand), but none of them continuous (if that ...
1 vote
60 views

### How to deal with missing values that are supposed to be missing?

I am trying to predict loan defaults with a fairly moderate-sized dataset. I will probably be using logistic regression and random forest. I have around 35 variables and one of them classifies the ...
• 13
37 views

### How to compute the Gini index, the entropy and the classification error from a decision tree?

How to find the Gini index, the entropy, and the classification error for each node of the tree in the figure below. Please help me to compute them.
• 21
4 views

### Neural Networks that can mimic Decision Trees

tl;dr: Are there neural networks (specific activation functions, set up of layers ...?) that can mimic fairly well decision trees/if statements? === I am trying to build a model from some simulated ...
• 21
13 views

### Resultant entropy of the target feature example

I'm confused by an example I have come across on entropy. In a decision tree, we have after a split on some particular feature, the following subset of our training data. What is the resultant ...
25 views

### Why my models have a pretty high accuracy with a small training dataset?

I was wondering why my models (decision tree, svm, random forest) behave like that, with "high" accuracy on a small training dataset. Is it a sign of overfitting? The graph represents the ...
35 views

### Decision Tree: SPRINT vs SLIQ?

I found different types of decision tree, for example SPRINT and SLIQ methods. Both methods are used for classification problem, use Gini Index for the feature selection and follow the procedure (...
• 411
26 views

### How to make TensorFlow Decision Forests models compatible with scikit-learn?

I am trying to create an ensemble using a tensorflow_decision_forests.keras.CartModel (from TensorFlow Decision Forests) as the ...
1 vote
18 views

### Not perfect accuracy when overfitting

Given a dataset and a decision tree that can be as depth as it wants, if you train the tree with all the dataset and then you test it against the same dataset and you get an accuracy that is not 100%, ...
43 views

### What kind of decision tree are used in random forest?

Read some documentation (for example) I know that there are many types of decision tree (Cart, ID3 and so on). I also know that Random Forest is a particolar algorithm that use a set of decision tree. ...
• 411
1 vote
15 views

### Regression tree cross validation confusing results

Setup I have implemented regression trees in go: full repository Using the full dataset and cost-complexity pruning, I get the following alphas and corresponding average residual (again, against the ...
12 views

### Is there a random forest env (sci-kit, TFDF, R, etc) that has an implementation for multi-output regression?

It is easy to adapt the idea of tree based linear regression to perform logistic regression: The decision boundaries of the tree divide the space of independent variables into hyper-cubes, and each ...
15 views

### Policy from an ensemble tree model?

I'm trying to output a policy (pdf on a fixed number of finite labels) from a Random Forest model. What is the best way to achieve this? I'm thinking that I can just normalize the votes from all the ...
200 views

### How Decision Tree Classifier works? [closed]

In particular i am using SKLearn with class DecisionTreeClassifier. I would really like to understand how the tree build itself ...
16 views

### Struggling with decision tree classification SKLearn and MultiLabelBinarizer [duplicate]

So i have training data like this: ...
7 views

### Extending a classifier with specialised features

Let's say we have an app and a classifier (GBDT) predicting whether a user is a good user or bad (whatever that means) based on generic signals that every user has like profile fields, how long they ...
46 views

### Low accuracy on the test set

I have a dataset with 16 features and 32 class labels, which shows the following behavior: Neural network classification: high accuracy on train 100%, but low accuracy on the test set 3% (almost like ...
• 163
31 views

### Does it make sense to add a new calculated column for dates/duration?

I'm using a Random Forest Classifier on some data, and I have two date field, StartDate and EndDate. Does it make sense to ...
• 1,248
10 views

### Best way to manually traverse hierarchical decision trees in R

I am currently building an application that allows users interact with tree models to generate classification rules that fit their specified parameters. The idea is simple: given any set of rules ...
• 1,621
21 views

### Simple way to parse the rules of a Tree model

Problem Description I am currently working with simple decision tree models to generate rules to classify respondents of a survey. The output of each model is "translated" to the complete ...
• 1,621
28 views

### Give more weight to features based on distribution plot

I have a task to predict a binary variable purchase, their dataset is strongly imbalanced (10:100) and the models I have tried so far (mostly ensemble) fail. In ...
• 101
32 views

### Simple CART model example

My goal is to test Decision tree to regression model. My data is like below(python dataframe). There are 2 features F1 and F2. And there is label which is number. How to make CART model from this ...
• 105
42 views

### Getting entropy in decision trees more than 1

My decision tree entropy is coming more than 1 when I'm calculating it manually. Not sure if there's some calculation error. Trying it on the Iris dataset. If I split on sepal length at 6.5 cm, my ...
38 views

### Why -2 is seen in supervised binning using decision tree?

I have a continuous variable called salary, age etc and output variable as loan_status Instead of me choosing the cut off points for salary and age bins , I used Decision Tree to compute the bins ...
• 2,129
478 views

### Are linear models better when dealing with too many features? If so, why?

I had to build a classification model in order to predict which what would be the user rating by using his/her review. (I was dealing with this dataset: Trip Advisor Hotel Reviews) After some ...
• 151
1 vote
33 views

### Understanding feature_parallel distributed learning algorithm in LightGBMClassifier

I want to understand feature_parallel algorithm in LightGBMClassifier. It describes how it is done traditionally and how LightGBM...
81 views

### SKLearn decisionTreeClassifier does not handle sparse or categorical data

Is there a way in fitting a decisionTreeClassifier in SKLearn to sparse tuples? The data that I have is based on about 100 features, but only a few of them are ever used to make the decision. ...
• 186
79 views

### Interpreting 'values' of a Decision Tree

I am trying to interpret my decision tree here which was resulted as a part of pre-pruning- I am trying to understand why the values in my nodes are in decimal places. Ideally, they should represent ...
• 23
19 views

### Approximating linear and non-linear functions using 1-depth trees

I'm taking a course in statistical learning and we learned about tree models, namely decision trees. Now, I'm preparing for the test and there are two questions from past exams: Given $y=x_1+2x_2$ ...
26 views

### Where can I find the original CART(classification and regression trees) published paper?

I was trying to find the original CART paper. I found papers like https://www.researchgate.net/publication/227658748_Classification_and_Regression_Trees which experimented on CART but was unable to ...
12 views

### Optimize parametric Log-Likelihood with a Decision Tree

There are some objects with features, and the target is parametric estimation for some model-based variables. Model parameters $\theta_j$ are obtained by maximizing log-likelihood. \$LL = \sum_{i \in ...
19 views

### R: Handling tags in R rpart

I'm new to regression trees and wanted some advice. I'm working on a data set in which a few of the columns contains tag-like information (basically tags separated by commas). My goal is to create a ...
77 views

### How to control a decision tree?

This is my R script for a decision tree: ...
• 411
26 views

### How to drawing a decision tree?

This is my script for a decision tree in R: ...
• 411
68 views

### Tree based Classifiers with Label Encoder and One Hot Encoder [duplicate]

I m working with Tree-based classifiers in scikit-learn - Decision Trees and Random Forest, for a data classification use case, and the feature set is a mix of both categorical (majority) and ...
• 101
21 views

### Error with decision tree prediction

I write this script in R about decision tree. ...
• 411
1 vote
27 views

### Reasons for a model predicting probability of being class 1 at x value

All, This is a general question. I have a binary classification which predicts if someone is rich or not. I had a question from someone asking that if the probability someone is rich is 0.6 and ...
• 466
15 views

### Looking for an algorithm to perform classification on multivariate grouped time series

I will be grateful for any help. I have multivariate time series, where every one of them has an unique ID. Also, there is a variable giving information about the trend type of the ID from a point of ...
20 views

### Handling repeating data from different individuals

I have a dataset that has some unique values but also includes information from multiple individuals that are repeating, meaning they are describing the same attributes and can have the same or ...
21 views

### How to apply post pruning methods to ID3 decision tree

I am developing an ID3 decision tree implementation that feature post-pruning and classification. The program below constructs the decision tree. ...
• 159
11 views

### Decision Trees and SHAP Values

I've recently been using some (optimal) decision trees methods in R, such as 'evtree' and 'iai.' Both of these provide really nice interpretable plots. And out of the 12 covariates I have in my model, ...
• 193