# Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

662 questions
Filter by
Sorted by
Tagged with
20 views

### How do I find entropy of features having numerical data? [duplicate]

I'm a newbie and I'm writing a decision tree from scratch using entropy and information gain. I understand that entropy is the measure of impurity of a data set and also calculated entropy for ...
200 views

### Value on Decision Tree plot

After plotting a sklearn decision tree I check what it says in each box and there is one feature "value" that I am not sure what it refers. The first line will be the column and the value where it ...
42 views

### Decision tree with multiple outputs

I have a sample with 10 independent variables (X1, X2, X3 ....), and multiple output labels (y1, y2, y3). Here y1 will depend on X1, X2 y2 will depend on X3, X4 and so on. y1, y2, y3 might or might ...
949 views

### What is the hypothesis space of decision tree learning?

Could you please explain what the hypothesis space for decision tree learning look like? And what is the cardinality of this space?
88 views

### Upper bound on size of sample set for decision trees

Say I have an instance space with 4 features and I know that a decision tree with 8 nodes can represent the target function I want to learn. I want to give an upper bound on the size of the sample set ...
28 views

### Analysis of Alternating Decision Tree on Weka

I am applying the AD Tree algorithm & this is the tree visualization of the output: I can't understand the values in the decision nodes (-0.4,0.541,-0.882...), How are these calculated? & how ...
16 views

### Is it possible for decision trees to consider less features than in my training set? [duplicate]

I was looking at the decision tree algorithm and I wondered that for example if the training set has 20 features but only 5 features are important and classification can be done by using only them ...
1k views

### Dealing with categorical variables in Isolation Forest

Isolation Forest is widely used when dealing with outlier/anomaly detection when we have no labels. The theory behind is that making random split at random points and counting how many splits you do ...
1k views

### plotting a decision tree based on gridsearchcv

i was trying to plot the decision tree which is formed with GridSearchCV, but its giving me an Attribute error. ...
361 views

### Negative value in information gain calculation through gini index

I am trying to determine the root node for the decision tree on given data annual income target variable has been renamed as ...
64 views

### Discretisation Using Decision Trees

I'm new to the machine learning and working on a supervised classification problem. I used discretization process to transform continuous variables into discrete variables. So I followed this ...
300 views

### How to Validate Decision Tree model by using *statistical tests*?

I'm reading sklearn Decision Trees reference page. In the advantages section, it is mentioned that 'Possible to validate a model using statistical tests. That makes it possible to account for the ...
449 views

### What is the intuitive meaning of "leaf weight" in xgboost

I looked through Tianqi Chen's presentation, but I'm struggling to understand the details of what the leaf weights are, and I would appreciate if someone could help clarify my understanding. To put ...
297 views

I was working on a project and was trying to validate my decisions. I wondered why would I want to use a decision tree over more powerful algorithms like random forest or Gradient boosting machine ...
280 views

### Pruning in Decision trees

Following is what I learned about the process followed during building and pruning a decision tree, mathematically (from Introduction to Machine Learning by Gareth James et al.): Use recursive binary ...
14 views

### I want to replace XGBRegressor with a simple model to make feature selection

I will make some for loop on to select the best features by my Data frame is big 10M row and about 50 columns so if i replaced xgb with a single Decision tree would it be the same results for the best ...
43 views

### How to Split Continuous Labelled Data?

I've started studying decision trees, and I noticed that the examples online used categorical features to split the data at each node. I'm working on data sets with a binary classification output and ...
33 views

### Do I need to transform time with sin/cos if I'm using decision tree algorithms?

According to this post, the time on a 24-hour clock should be decomposed into separate periodic components: https://ianlondon.github.io/blog/encoding-cyclical-features-24hour-time/ before feeding it ...
127 views

### ExtraTreeClassifier does not handle missing values

I am using sklearn.tree.ExtraTreeClassifier. It does not handle missing value in training data. All tree-based algorithms handle missing value internally. So, is ...
176 views

### Gini index as a labeling strategy for leaf nodes

Can we use the gini index to assign a class to a leaf node? If yes how? As far as I understand the gini index can only be used as a splitting metric.
216 views

### Impurity measures in decision trees

I have recently stepped into impurity based criteria for decision trees and I was just wondering why do we really need an impurity based criteria model such as the Gini index? What if we could simply ...
1k views

### XGBoost and Random Forest: ntrees vs. number of boosting rounds vs. n_estimators

So I understand the main difference between Random Forests and GB Methods. Random Forests grow parallel trees and GB Methods grow one tree for each iteration. However, I am confused on the vocab used ...
51 views

### How to decide who to market? Clustering or Decision Tree?

I am working with a dataset that has enough observations and ~ 10 variables, half of the variables are numeric another half of the variables are categorical with 2-3 levels (demographics) one ID ...
1k views

### Decision Trees and Feature Selection

I'm trying to experiment with the performance of different machine learning algorithms before and after applying feature selection. I tested SVM, Random Forest, KNN, Linear Regression, and, Decision ...
38 views

### Different Decision Tree pruning method

I am trying to learn different pruning methods for decision trees. I have put together a list of methods below. Reduced Error Pruning Cost Complexity pruning Minimum error pruning Pessimistic Error ...
444 views

### Decision tree too small

I have a data set of 2300 entries, with 5 variables one of them the dependent variable which is binary. I fitted a decision try using the rpart function in R over ...
187 views

### What is the scalability of linear regression and decision trees?

Recently I'm studying machine learning algorithms among them linear regression and decision tree so I have a question regarding the scalability of both algorithms. Can anyone provide what is the ...
54 views

### How could a neural network classifer for multilclass problem classify only in one class when a decision tree is more balanced and accurate?

I want to create a classifier for a data frame that has four classes. Each line can only have one class. I have two predictive models: a neural network and a tree classifier. But they put everyone in ...
45 views

### How to find the dependent variables from a dataset?

I am stuck at where how can I get the most dependent variables based on the mean I have this dataset and when I try to: ...
74 views

### upsampling imbalanced dataset in decision tree

I have a imbalanced dataset with 3 output labels with one class with 98 percent and other two classes with 1 percent each. I need to run decision tree on this dataset. Should i be upsampling this ...
61 views

### treeExplainer algorithm intuition

I'm reading the paper about the treeExplainer; the pseudo-code of Algorithm 1 is a bit cryptical as most of the variables are not even defined (same with sampling and all details involved). Is there a ...
1k views

### Why do decision trees have low accuracy?

It seems to be generally acknowledged that decision trees have low prediction accuracy. Is there a concise explanation for why they have low accuracy? I've read this so much, I've accepted it to be ...
2k views

### what should i do if my target variable is categorical when using decision tree? (many categorical variables)

all, i'm trying to classify a set of features to belong to a particular company (my dependent variable). my independent variables are a mixture of continuous and categorical features. my data-set ...
116 views

### I intend to do classification modelling, but my target variable has only one value

Currently I have a dataset and I am trying to predict whether someone will default on their bank loan. The dataset is quite tricky. It covers those who have defaulted in the past, but is also ...
134 views

### Numeric variables in Decision trees

If we have numeric variable, decision trees will use < and > comparisons as splitting criteria. Lets consider this case : ...
45 views

### Traditional ML Model or Deep Learning for ~200-300 samples?

Good morning all! I'm working on a resume parser that is integrated with an RPA (robotic process automation) platform. The robot has OCR to extract text from a PDF resume, and it supplies the ...
41 views

### Difference between shap values and feature contributions

I always found both concepts a bit confusing since they are quite similar. Would someone provide clear example where to apply each? Shap values ref: https://towardsdatascience.com/explain-your-model-...
54 views

### Custom Decision Function for Custom Outlier Detection Algorithm

I have built a custom algorithm for semi-supervised anomaly detection and here is my output example as following with probability threshold set to 0.05 and 1 = outlier, 0 = inlier: ...
572 views

### Decision Trees change result at every run, how can I trust of my results?

Given a database, I split the data in train and test. I want to use a decision-tree classifier (sklearn) for a binary classification problem. Considering I already found the best parameters for my ...
423 views

### Extract features from Decision tree leaf nodes

Recently came across a coursera course on "How to win Kaggle competitions" where they explain how we can engineer a categorical feature from each leaf node of the decision tree. [Video Link][1] I ...
37 views

### How does the construction of a decision tree differ for different optimization metrics?

I understand how a decision tree is constructed (in the ID3 algorithm) using criterion such as entropy, gini index, and variance reduction. But the formulae for these criteria do not care about ...
302 views

### Can one property name be used twice in the same branch of a DecisionTreeRegressor?

I am using this dataset for the analysis (Generated using make_regression of sklearn library) I was trying to learn the DecisionTreeRegression algorithm of sklearn library. I used the following code ...
22 views

### Serializing a trained classification model into a set of actionable insights

I'm looking for ways to convert a trained classification model into a list of insights based on the resulting parameters of the model. To make an example, let's assume we trained a decision tree to ...
49 views

### Random forest mode scoring

We are using random forest algorithm but having some trouble understanding the scoring method it uses. take for example the following CM of the test set: ...
659 views

### XGBOOST - different result between train_test_split and manually splitting

I am trying to train XGBOOST model. X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=43, stratify=y) when I'm using ...
375 views

### Interpretable xgboost - Calculate cover feature importance

When trying to interpret the results of a gradient boosting (or any decision tree) one can plot the feature importance. There are same parameters in the xgb api such as: weight, gain, cover, ...
42 views

### Multiclass XGBoost train with num classes = 2

I have a tagged csv file with 5 calsses. I accidentally trained am XGBOOST model with this input but forgot to change the num_classes to 5, but instead it was still 2. The model I received seems to ...
976 views

### Python and GridSearchCV how to eliminate input contains NaN error when using cross validation and decision tree classifier?

I am trying to do cross validation on Decision tree classifier for kaggle's titanic dataset. The first step after cleaning data is to split into train and test sets: ...