Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
1 vote
1 answer
29 views

How to use labels to fit several thresholds in a simple decision rule?

I have a binary labelled dataset with numeric features. I want to create a "business rule" of the type y = x1 > t1 and x2 > t2 and x3 > t3. ...
hipoglucido's user avatar
  • 1,170
1 vote
1 answer
36 views

Can the product of tree regressions be represented by a single tree?

Assume that we have two separate tree regressions. I'm interested in understanding whether the product of tree regressions can be represented by a single tree. Would this be possible?
TFT's user avatar
  • 135
4 votes
2 answers
636 views

Loss for ordered multi class data in classification

Assume data which is labeled $y_i \in \left\{ 1, 2, 3, \ldots, 9, 10\right\}$. Assume the labels are ordered, namely, given $y_i = 10$ to estimate $\hat{y}_{i} = 1$ is much worse than $\hat{y}_{i} = ...
Eric Johnson's user avatar
0 votes
1 answer
3k views

Decision tree vs logistic regression feature importances

I have trained Logistic regression and decision tree in skearn on the same standardized dataset (binary classification). Top important coefficients for the decision tree are (sorted by ...
Arseniy Maryin's user avatar
3 votes
1 answer
776 views

Why am I getting the exact same results with both a Logistic Regression and Decision Tree Classifier?

I am working on a binary classification problem and am using sklearn's logistic regression model and decision tree classifier. Somehow I am getting the exact same results and accuracy score on both. I ...
RandomGuy's user avatar
0 votes
0 answers
60 views

My overfitted decision tree regressor gives better result than pre-pruned tree?

I create a decision tree regressor without giving any parameters. The resulting tree has 6255 leaf nodes (out of 6348 entries of train set) and depth of 39. Most probably it has overfitted. But its ...
Akrobeto's user avatar
1 vote
1 answer
76 views

N-ary decision tree with categorical features

I want to build an n-ary decision tree with categorical features. I am using ordinary ID3 algorithm to build a tree. Lets take the next dataset as a training dataset for building a decision tree: ...
dzi's user avatar
  • 111
0 votes
1 answer
287 views

How can I train a decision tree constrained to have number of decision nodes = tree depth?

In order to make a classifier dead easy to understand/interpret, I want to classify tabular data (with n columns) according to a set of nested rules, with the ...
Davide Fiocco's user avatar
0 votes
1 answer
28 views

How do I use one hot encoding with 240 nominal variables and each with equal occurrence?

The method I saw that's generally used to deal with large # of nominal variables is to keep the most frequent variables and introduce a new "other" category. But that's not possible with my ...
learner's user avatar
0 votes
1 answer
108 views

How to properly use regression / tree based models for time-series data

Regression/tree-based models appear to treat each prediction as a memoryless process, namely given a feature vector $\hat{x}_i$, predict $y_{i+1}$, but previous states $\hat{x}_{i-1}$, $\hat{x}_{i-2}, ...
ron burgundy's user avatar
0 votes
2 answers
60 views

Can Gradient Descent boosted decision trees miss the forest for the trees?

My understanding of this stuff is pretty basic so my semantics may be off, but bare with me. XGBoost and other gradient descent packages make the best possible split of the data right off the bat. ...
helloimgeorgia's user avatar
0 votes
1 answer
812 views

Splitter in decision trees in sklearn implementation

I am very confused about how decision trees select features and threshold within each feature to do the split. I totally understand the different splitting metrics (Gini index and so on) used and how ...
AAA's user avatar
  • 35
0 votes
1 answer
166 views

how to compute the possible number of splits in decision tree?

In the following dataset, if we want to include just two variables, STORE and PctDiscMM, in a classification tree model, what is the possible number of first splits? ...
ebrahimi's user avatar
  • 1,307
1 vote
4 answers
626 views

Draw a decision tree with depth 2 that is consistent with the data

I am trying to come up with a solution to this for an exam preparation but cant come up with anything, dont know how to tackle it... if i use information gain the depth increases beyond 2. What would ...
fearloathing121's user avatar
2 votes
1 answer
63 views

Model from an aggregate

I’m in a place where we’re unable to train models on data due to GDPR. What I want is to predict people getting a job (y) given (x,x,x,x…) their employment type working full time or part time, work ...
SamTheGoodOne's user avatar
2 votes
1 answer
840 views

Online Learning/Continual Learning for tree-based Algorithms

Every example I come across any kind of iterative learning on Random Forest/XGBoost/LightGBM, it just continuously grows the number of estimators for new batches of data by ...
OliverHennhoefer's user avatar
0 votes
1 answer
2k views

I am facing the error of DecisionTreeregressor object has no attribute n_features

...
gaurav's user avatar
  • 1
1 vote
0 answers
8 views

Precision and AUROC for which class values

I am a newbie in reading research paper and implementing it by myself. I went through the paper Breast Cancer Survival Prediction from Imbalanced Dataset with Machine Learning Algorithms. Can anyone ...
Encipher's user avatar
  • 361
1 vote
1 answer
352 views

Decision tree implementation in python that correctly handles categorical variables

Is there a Decision tree implementation in python that correctly handles categorical covariates? By "correct" I mean that it is able to send any subset of category levels down one daughter ...
Iyar Lin's user avatar
  • 799
1 vote
0 answers
31 views

ML Model that doesnt average/penalize extreme values

I have a 20k dataset, and a couple hundred of those lines are extreme values and 10 of them or so are even extremer values. But they are correct and have a unique tag, so when that tag comes up I am ...
Jroc561's user avatar
  • 11
0 votes
0 answers
41 views

Is it possible to extract precise decision boundaries from a random forest for a multiclass classification?

I have a random forest (argmax post-processor) with 3 trees and 10 input features. The final outcome of the random forest is either true or false depending on the combination of the feature values. Is ...
giantjenga's user avatar
0 votes
2 answers
131 views

What is the implication of having features with less variation in a tree based model?

I'm training a tree-based model (e.g. xgb). I have some features with more than 90% values constant. Does it add value to the model since the variation in the data is minimal?. What would be the ...
NAS_2339's user avatar
  • 243
0 votes
1 answer
120 views

The best Python library to build decision tree on binary inputs

Please, could you advise me the best Python library for the following problem. I have 60 binary input variables and a binary output variable. There are 10 000 – 20 000 training examples. I want to ...
PierreVanStulov's user avatar
2 votes
2 answers
240 views

Are feature importances of ensemble methods sensible interpretable?

As mentioned in the question, it is easy to interpret the meaning of features in algorithms like simple decision trees. But in the case of ensemble methods that are known to average/modify features, ...
JAdel's user avatar
  • 23
0 votes
1 answer
158 views

Show how to obtain decision tree will classify the test instance <sunny,mild,normal,weak>?

Given the question, The decision tree for this is, But unable to predict the sample. Can anyone help me in this question. TIA
XYZ's user avatar
  • 121
0 votes
0 answers
71 views

Is it possible to embed a neural network layer into decision tree/random forest?

I want to do a classification task. I designed a customed layer for it. I also want to try decision tree/random forest, but as far as I know there is no way to embed my layer into a decsion tree/...
user900476's user avatar
1 vote
0 answers
108 views

Error from XGBoost missing data handling

I have a regression problem with a very large dataset >50 million rows, 81 features and 1 target, all positive float values unevenly distributed between 0 - 1 million. I've trained an XGBoost model ...
lexan55's user avatar
  • 36
2 votes
2 answers
458 views

Which Model for predicting flight delays is appropriate except Random Forest and Decision Tree? (Monte Carlo?)

Im studying M.Sc Data Science and in the module "Decision Support Systems" me and my group have to make a presentation. Our Proposal is the following: Background With generally high demand ...
wayne's user avatar
  • 21
0 votes
0 answers
14 views

Decision tree question in R

After training and testing the decision tree model, it always gives me the same outcome on any given data. Im talking about a binary classification yes or no. Basically when I do predict(fit_model, ...
Dom's user avatar
  • 1
2 votes
0 answers
20 views

Obtaining threshold based rules for classification problem

Suppose there are X1...Xn numerical variables predicting a target variable Y (0 or 1) Objective: to obtain the best possible thresholds and combinations of X1...Xn that can predict Y Example: (X1>...
Sunit Gautam's user avatar
0 votes
1 answer
40 views

GridSearch multiplying the number of trees in XGboost?

I'm having an issue: after running an XGboost in a HalvingGridSearchCV, I receive a certain number of estimators (50 for example), but the number of trees is constantly being multiplied by 3. I don't ...
Cosapocha's user avatar
2 votes
1 answer
83 views

How to generate a rule-based system based on binary data?

I have a dataset where each row is a sample and each column is a binary variable. The meaning of $X_{i, j} = 1$ is that we've seen feature $j$ for sample $i$. $X_{i, j} = 0$ means that we haven't seen ...
greenButMellow's user avatar
0 votes
2 answers
96 views

Random Forest plot standardized

For a data science project, I first used a standardized scaler on data in python, ran random forest then plotted the tree. However, the values of the decisions are in their standardized form. How do I ...
firedonut123's user avatar
1 vote
2 answers
32 views

How do I design a random forest split with a "not sure" category?

Let's say I have data with two target labels, A and B. I want to design a random forest that has three outputs: A, B and Not sure. Items in the Not sure category would be a mix of A and B that would ...
cjm2671's user avatar
  • 284
0 votes
1 answer
56 views

How does ExtraTrees (Extremely Randomized Trees) learn?

I'm trying to understand the difference between random forests and extremely randomized trees (https://orbi.uliege.be/bitstream/2268/9357/1/geurts-mlj-advance.pdf) I understand that extratrees uses ...
cjm2671's user avatar
  • 284
1 vote
0 answers
50 views

Machine learning frameworks for tree-based models

Background: Its well known that Pytorch and TensorFlow are currently the most used frameworks for Deep Learning (DL) research. As far as I know, most researchers (applied or theoretical) that ...
Enk9456's user avatar
  • 105
1 vote
1 answer
1k views

forcing decision tree use specific features first

My goal it to force some feature used firstly to split tree. Below, the function splitted tree using feature_3 first. For instance, is there a way to force to use feature_2 first instead of feature_3 ?...
Soon's user avatar
  • 115
0 votes
2 answers
3k views

What is the max number of leaf nodes in a classifcation decision tree?

Let's assume that we have n observations and p predictors and we have in a n>>p situation. All predictors are binary. What is the max number of leaf nodes that we can have in the tree? and what ...
giacomo venturelli's user avatar
0 votes
1 answer
30 views

Identifying subsets of values significant to the total sum

Imagine a set of products in a store, with all the different attributes assigned to them - some of these hierarchical (e.g. categories), and some not (e.g. brand), but none of them continuous (if that ...
curl-up's user avatar
1 vote
1 answer
123 views

How to deal with missing values that are supposed to be missing?

I am trying to predict loan defaults with a fairly moderate-sized dataset. I will probably be using logistic regression and random forest. I have around 35 variables and one of them classifies the ...
IcarusX's user avatar
  • 13
0 votes
1 answer
388 views

How to compute the Gini index, the entropy and the classification error from a decision tree?

How to find the Gini index, the entropy, and the classification error for each node of the tree in the figure below. Please help me to compute them.
Nezuko's user avatar
  • 21
0 votes
0 answers
17 views

Resultant entropy of the target feature example

I'm confused by an example I have come across on entropy. In a decision tree, we have after a split on some particular feature, the following subset of our training data. What is the resultant ...
NotationStation's user avatar
0 votes
1 answer
38 views

Why my models have a pretty high accuracy with a small training dataset?

I was wondering why my models (decision tree, svm, random forest) behave like that, with "high" accuracy on a small training dataset. Is it a sign of overfitting? The graph represents the ...
rrnco1234's user avatar
1 vote
2 answers
965 views

Decision Tree: SPRINT vs SLIQ?

I found different types of decision trees, for example, SPRINT and SLIQ methods. Both methods are used for the classification problems, using Gini Index for the feature selection and following the ...
Inuraghe's user avatar
  • 481
1 vote
1 answer
97 views

Not perfect accuracy when overfitting

Given a dataset and a decision tree that can be as depth as it wants, if you train the tree with all the dataset and then you test it against the same dataset and you get an accuracy that is not 100%, ...
Carlos Navarro Astiasarán's user avatar
0 votes
2 answers
330 views

What kind of decision tree are used in random forest?

Read some documentation (for example) I know that there are many types of decision tree (Cart, ID3 and so on). I also know that Random Forest is a particolar algorithm that use a set of decision tree. ...
Inuraghe's user avatar
  • 481
1 vote
0 answers
18 views

Regression tree cross validation confusing results

Setup I have implemented regression trees in go: full repository Using the full dataset and cost-complexity pruning, I get the following alphas and corresponding average residual (again, against the ...
wheresmycookie's user avatar
2 votes
2 answers
344 views

How Decision Tree Classifier works? [closed]

In particular i am using SKLearn with class DecisionTreeClassifier. I would really like to understand how the tree build itself ...
baltiturg's user avatar
  • 143
0 votes
0 answers
17 views

Struggling with decision tree classification SKLearn and MultiLabelBinarizer [duplicate]

So i have training data like this: ...
baltiturg's user avatar
  • 143
0 votes
1 answer
337 views

Low accuracy on the test set

I have a dataset with 16 features and 32 class labels, which shows the following behavior: Neural network classification: high accuracy on train 100%, but low accuracy on the test set 3% (almost like ...
Albert's user avatar
  • 163

1
2
3 4 5
16