Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
0
votes
0answers
20 views

How do I find entropy of features having numerical data? [duplicate]

I'm a newbie and I'm writing a decision tree from scratch using entropy and information gain. I understand that entropy is the measure of impurity of a data set and also calculated entropy for ...
0
votes
1answer
200 views

Value on Decision Tree plot

After plotting a sklearn decision tree I check what it says in each box and there is one feature "value" that I am not sure what it refers. The first line will be the column and the value where it ...
1
vote
2answers
42 views

Decision tree with multiple outputs

I have a sample with 10 independent variables (X1, X2, X3 ....), and multiple output labels (y1, y2, y3). Here y1 will depend on X1, X2 y2 will depend on X3, X4 and so on. y1, y2, y3 might or might ...
1
vote
1answer
949 views

What is the hypothesis space of decision tree learning?

Could you please explain what the hypothesis space for decision tree learning look like? And what is the cardinality of this space?
1
vote
1answer
88 views

Upper bound on size of sample set for decision trees

Say I have an instance space with 4 features and I know that a decision tree with 8 nodes can represent the target function I want to learn. I want to give an upper bound on the size of the sample set ...
1
vote
1answer
28 views

Analysis of Alternating Decision Tree on Weka

I am applying the AD Tree algorithm & this is the tree visualization of the output: I can't understand the values in the decision nodes (-0.4,0.541,-0.882...), How are these calculated? & how ...
0
votes
0answers
16 views

Is it possible for decision trees to consider less features than in my training set? [duplicate]

I was looking at the decision tree algorithm and I wondered that for example if the training set has 20 features but only 5 features are important and classification can be done by using only them ...
3
votes
0answers
1k views

Dealing with categorical variables in Isolation Forest

Isolation Forest is widely used when dealing with outlier/anomaly detection when we have no labels. The theory behind is that making random split at random points and counting how many splits you do ...
0
votes
1answer
1k views

plotting a decision tree based on gridsearchcv

i was trying to plot the decision tree which is formed with GridSearchCV, but its giving me an Attribute error. ...
1
vote
1answer
361 views

Negative value in information gain calculation through gini index

I am trying to determine the root node for the decision tree on given data annual income target variable has been renamed as ...
1
vote
1answer
64 views

Discretisation Using Decision Trees

I'm new to the machine learning and working on a supervised classification problem. I used discretization process to transform continuous variables into discrete variables. So I followed this ...
3
votes
2answers
300 views

How to Validate Decision Tree model by using *statistical tests*?

I'm reading sklearn Decision Trees reference page. In the advantages section, it is mentioned that 'Possible to validate a model using statistical tests. That makes it possible to account for the ...
2
votes
1answer
449 views

What is the intuitive meaning of "leaf weight" in xgboost

I looked through Tianqi Chen's presentation, but I'm struggling to understand the details of what the leaf weights are, and I would appreciate if someone could help clarify my understanding. To put ...
1
vote
3answers
297 views

Advantages and disadvantages of using classification tree

I was working on a project and was trying to validate my decisions. I wondered why would I want to use a decision tree over more powerful algorithms like random forest or Gradient boosting machine ...
0
votes
1answer
280 views

Pruning in Decision trees

Following is what I learned about the process followed during building and pruning a decision tree, mathematically (from Introduction to Machine Learning by Gareth James et al.): Use recursive binary ...
0
votes
0answers
14 views

I want to replace XGBRegressor with a simple model to make feature selection

I will make some for loop on to select the best features by my Data frame is big 10M row and about 50 columns so if i replaced xgb with a single Decision tree would it be the same results for the best ...
1
vote
0answers
43 views

How to Split Continuous Labelled Data?

I've started studying decision trees, and I noticed that the examples online used categorical features to split the data at each node. I'm working on data sets with a binary classification output and ...
1
vote
0answers
33 views

Do I need to transform time with sin/cos if I'm using decision tree algorithms?

According to this post, the time on a 24-hour clock should be decomposed into separate periodic components: https://ianlondon.github.io/blog/encoding-cyclical-features-24hour-time/ before feeding it ...
0
votes
0answers
127 views

ExtraTreeClassifier does not handle missing values

I am using sklearn.tree.ExtraTreeClassifier. It does not handle missing value in training data. All tree-based algorithms handle missing value internally. So, is ...
2
votes
1answer
176 views

Gini index as a labeling strategy for leaf nodes

Can we use the gini index to assign a class to a leaf node? If yes how? As far as I understand the gini index can only be used as a splitting metric.
0
votes
1answer
216 views

Impurity measures in decision trees

I have recently stepped into impurity based criteria for decision trees and I was just wondering why do we really need an impurity based criteria model such as the Gini index? What if we could simply ...
6
votes
2answers
1k views

XGBoost and Random Forest: ntrees vs. number of boosting rounds vs. n_estimators

So I understand the main difference between Random Forests and GB Methods. Random Forests grow parallel trees and GB Methods grow one tree for each iteration. However, I am confused on the vocab used ...
2
votes
2answers
51 views

How to decide who to market? Clustering or Decision Tree?

I am working with a dataset that has enough observations and ~ 10 variables, half of the variables are numeric another half of the variables are categorical with 2-3 levels (demographics) one ID ...
3
votes
2answers
1k views

Decision Trees and Feature Selection

I'm trying to experiment with the performance of different machine learning algorithms before and after applying feature selection. I tested SVM, Random Forest, KNN, Linear Regression, and, Decision ...
1
vote
0answers
38 views

Different Decision Tree pruning method

I am trying to learn different pruning methods for decision trees. I have put together a list of methods below. Reduced Error Pruning Cost Complexity pruning Minimum error pruning Pessimistic Error ...
1
vote
3answers
444 views

Decision tree too small

I have a data set of 2300 entries, with 5 variables one of them the dependent variable which is binary. I fitted a decision try using the rpart function in R over ...
0
votes
1answer
187 views

What is the scalability of linear regression and decision trees?

Recently I'm studying machine learning algorithms among them linear regression and decision tree so I have a question regarding the scalability of both algorithms. Can anyone provide what is the ...
1
vote
1answer
54 views

How could a neural network classifer for multilclass problem classify only in one class when a decision tree is more balanced and accurate?

I want to create a classifier for a data frame that has four classes. Each line can only have one class. I have two predictive models: a neural network and a tree classifier. But they put everyone in ...
1
vote
2answers
45 views

How to find the dependent variables from a dataset?

I am stuck at where how can I get the most dependent variables based on the mean I have this dataset and when I try to: ...
-1
votes
1answer
74 views

upsampling imbalanced dataset in decision tree

I have a imbalanced dataset with 3 output labels with one class with 98 percent and other two classes with 1 percent each. I need to run decision tree on this dataset. Should i be upsampling this ...
2
votes
1answer
61 views

treeExplainer algorithm intuition

I'm reading the paper about the treeExplainer; the pseudo-code of Algorithm 1 is a bit cryptical as most of the variables are not even defined (same with sampling and all details involved). Is there a ...
1
vote
2answers
1k views

Why do decision trees have low accuracy?

It seems to be generally acknowledged that decision trees have low prediction accuracy. Is there a concise explanation for why they have low accuracy? I've read this so much, I've accepted it to be ...
1
vote
2answers
2k views

what should i do if my target variable is categorical when using decision tree? (many categorical variables)

all, i'm trying to classify a set of features to belong to a particular company (my dependent variable). my independent variables are a mixture of continuous and categorical features. my data-set ...
1
vote
2answers
116 views

I intend to do classification modelling, but my target variable has only one value

Currently I have a dataset and I am trying to predict whether someone will default on their bank loan. The dataset is quite tricky. It covers those who have defaulted in the past, but is also ...
4
votes
1answer
134 views

Numeric variables in Decision trees

If we have numeric variable, decision trees will use < and > comparisons as splitting criteria. Lets consider this case : ...
1
vote
1answer
45 views

Traditional ML Model or Deep Learning for ~200-300 samples?

Good morning all! I'm working on a resume parser that is integrated with an RPA (robotic process automation) platform. The robot has OCR to extract text from a PDF resume, and it supplies the ...
1
vote
0answers
41 views

Difference between shap values and feature contributions

I always found both concepts a bit confusing since they are quite similar. Would someone provide clear example where to apply each? Shap values ref: https://towardsdatascience.com/explain-your-model-...
1
vote
0answers
54 views

Custom Decision Function for Custom Outlier Detection Algorithm

I have built a custom algorithm for semi-supervised anomaly detection and here is my output example as following with probability threshold set to 0.05 and 1 = outlier, 0 = inlier: ...
1
vote
1answer
572 views

Decision Trees change result at every run, how can I trust of my results?

Given a database, I split the data in train and test. I want to use a decision-tree classifier (sklearn) for a binary classification problem. Considering I already found the best parameters for my ...
2
votes
1answer
423 views

Extract features from Decision tree leaf nodes

Recently came across a coursera course on "How to win Kaggle competitions" where they explain how we can engineer a categorical feature from each leaf node of the decision tree. [Video Link][1] I ...
1
vote
1answer
37 views

How does the construction of a decision tree differ for different optimization metrics?

I understand how a decision tree is constructed (in the ID3 algorithm) using criterion such as entropy, gini index, and variance reduction. But the formulae for these criteria do not care about ...
0
votes
1answer
302 views

Can one property name be used twice in the same branch of a DecisionTreeRegressor?

I am using this dataset for the analysis (Generated using make_regression of sklearn library) I was trying to learn the DecisionTreeRegression algorithm of sklearn library. I used the following code ...
3
votes
0answers
22 views

Serializing a trained classification model into a set of actionable insights

I'm looking for ways to convert a trained classification model into a list of insights based on the resulting parameters of the model. To make an example, let's assume we trained a decision tree to ...
2
votes
1answer
49 views

Random forest mode scoring

We are using random forest algorithm but having some trouble understanding the scoring method it uses. take for example the following CM of the test set: ...
5
votes
2answers
659 views

XGBOOST - different result between train_test_split and manually splitting

I am trying to train XGBOOST model. X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=43, stratify=y) when I'm using ...
2
votes
1answer
375 views

Interpretable xgboost - Calculate cover feature importance

When trying to interpret the results of a gradient boosting (or any decision tree) one can plot the feature importance. There are same parameters in the xgb api such as: weight, gain, cover, ...
0
votes
0answers
42 views

Multiclass XGBoost train with num classes = 2

I have a tagged csv file with 5 calsses. I accidentally trained am XGBOOST model with this input but forgot to change the num_classes to 5, but instead it was still 2. The model I received seems to ...
1
vote
2answers
976 views

Python and GridSearchCV how to eliminate input contains NaN error when using cross validation and decision tree classifier?

I am trying to do cross validation on Decision tree classifier for kaggle's titanic dataset. The first step after cleaning data is to split into train and test sets: ...
2
votes
1answer
109 views

Correlation based Feature Selection vs Feature Engineering

I'm a bit confused about the superiority of Feature Selection over Feature Engineering or vice versa. Let's say I just want to get the best possible performance on a couple of models like a neural ...
1
vote
1answer
26 views

Which decision tree model is used in "standard" random forest?

Is that CART? Why not using C5.0 tree? A perhaps more general question: Since C5.0 tree frequently have better performance than CART, why people still use CART to build random forest (or people ...

1
3 4
5
6 7
14