Skip to main content

Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
10 votes
5 answers
14k views

Why decision tree needs categorical variable to be encoded?

As per my intuition, decision trees should work better with categorical variables than with continuous variables. If this is the case, why is encoding needed on categorical variables? Can someone give ...
Mukesh K's user avatar
  • 101
31 votes
1 answer
33k views

How is a splitting point chosen for continuous variables in decision trees?

I have two questions related to decision trees: If we have a continuous attribute, how do we choose the splitting value? Example: Age=(20,29,50,40....) Imagine that we have a continuous attribute $f$...
WALID BELRHALMIA's user avatar
7 votes
2 answers
2k views

Is Label Encoding with arbitrary numbers ever useful at all?

From what I read online, there seems to be some confusion regarding the taxonomy and the terms used, so to avoid misunderstanding I'm going to define them here: Label Encoding - encoding a nominal ...
UchuuStranger's user avatar
2 votes
1 answer
729 views

Can a decision in a node of a decision tree be based on comparison between 2 columns of the dataset?

Assume the features in the dataframe are columns - A,B,C and my target is Y Can my decision tree have a decision node which looks for say, ...
Jerry's user avatar
  • 43
87 votes
6 answers
153k views

strings as features in decision tree/random forest

I am doing some problems on an application of decision tree/random forest. I am trying to fit a problem which has numbers as well as strings (such as country name) as features. Now the library, scikit-...
user3001408's user avatar
  • 1,005
11 votes
2 answers
4k views

Is max_depth in scikit the equivalent of pruning in decision trees?

I was analyzing the classifier created using a decision tree. There is a tuning parameter called max_depth in scikit's decision tree. Is this equivalent of pruning a decision tree? If not, how could I ...
Suhail Gupta's user avatar
9 votes
1 answer
5k views

When does decision tree perform better than the neural network?

I was experimenting with different modelling methods including KNN, Decision Trees, Neural Networks and SVN and trying to fit my data to see which works the best. To my surprise, the decision tree ...
Suhail Gupta's user avatar
8 votes
2 answers
7k views

Can a decision tree learn to solve a xOR problem?

I have read online that decision trees can solve xOR type problems, as shown in images (xOR problem: 1) and (Possible solution as decision tree: 2). My question is how can a decision tree learn to ...
lguerra's user avatar
  • 83
112 votes
9 answers
139k views

When should I use Gini Impurity as opposed to Information Gain (Entropy)?

Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)? Which metric is better to use in different scenarios while using decision trees?
Krish Mahajan's user avatar
26 votes
4 answers
97k views

How to make a decision tree with both continuous and categorical variables in the dataset?

Let's say I have 3 categorical and 2 continuous attributes in a dataset. How do I build a decision tree using these 5 variables? Edit: For categorical variables, it is easy to say that we will split ...
Sahil Chaturvedi's user avatar
13 votes
2 answers
30k views

How to normalize data for Neural Network and Decision Forest

I have a data set with 20000 samples, each has 12 different features. Each sample is either in category 0 or 1. I want to train a neural network and a decision forest to categorize the samples so that ...
Merlin1896's user avatar
10 votes
2 answers
20k views

Multicollinearity in Decision Tree

Can anybody please explain the affect of multicollinearity on Decision Tree algorithms (Classification and regression). I have done some searching but was not able to find the right answer as some say ...
deepguy's user avatar
  • 1,441
10 votes
2 answers
10k views

Why `max_features=n_features` does not make the Random Forest independent of number of trees?

Consider the following simple classification problem (Python, scikit-learn) ...
Jorge Leitao's user avatar
5 votes
2 answers
1k views

Decision Trees Should We Discard Low Importance Features?

I just started to work with feature selection. Let's say I have a decision tree model. I get its feature importances by tree.feature_importances_. In my model out ...
J.Smith's user avatar
  • 468
3 votes
1 answer
245 views

Make a random forest estimator the exact same of a decision tree

The idea is to make one of the trees of a Random Forest, to be built exactly equal to a Decision Tree. First, we load all libraries, fit a decision tree and plot it. ...
Carlos Mougan's user avatar
0 votes
1 answer
2k views

Decision tree not using all features from training dataset

I have built CART model using sklearn. I'm having total 6 features in training dataset and passing all of them in fit function. I've tested both criteria Gini and entropy. But whenever I plot tree ...
Scorpionk's user avatar
  • 199
40 votes
4 answers
38k views

Why do we need XGBoost and Random Forest?

I wasn't clear on couple of concepts: XGBoost converts weak learners to strong learners. What's the advantage of doing this ? Combining many weak learners instead of just using a single tree ? ...
John Constantine's user avatar
38 votes
5 answers
59k views

Are decision tree algorithms linear or nonlinear

Recently a friend of mine was asked whether decision tree algorithms are linear or nonlinear algorithms in an interview. I tried to look for answers to this question but couldn't find any satisfactory ...
user2966197's user avatar
36 votes
3 answers
37k views

Is it necessary to normalize data for XGBoost?

MinMaxScaler() in scikit-learn is used for data normalization (a.k.a feature scaling). Data normalization is not necessary for ...
user781486's user avatar
  • 1,425
22 votes
1 answer
33k views

XGBRegressor vs. xgboost.train huge speed difference?

If I train my model using the following code: ...
user1566200's user avatar
13 votes
4 answers
20k views

Interpreting Decision Tree in context of feature importances

I'm trying to understand how to fully understand the decision process of a decision tree classification model built with sklearn. The 2 main aspect I'm looking at are a graphviz representation of the ...
Tim Lindsey's user avatar
13 votes
1 answer
22k views

How max_features parameter works in DecisionTreeClassifier?

What is the parameter max_features in DecisionTreeClassifier responsible for? I thought it defines the number of features the ...
James Flash's user avatar
13 votes
3 answers
24k views

Unbalanced classes -- How to minimize false negatives?

I have a dataset that has a binary class attribute. There are 623 instances with class +1 (cancer positive) and 101,671 instances with class -1 (cancer negative). I've tried various algorithms (Naive ...
user798275's user avatar
12 votes
1 answer
32k views

XGBoost for binary classification: choosing the right threshold

I am working on a highly-imbalanced binary-labeled dataset, where number of true labels is just 7% from the whole dataset. But some combination of features could yield higher than average number of ...
Denis Kulagin's user avatar
12 votes
2 answers
6k views

Why neural networks do not perform well on structured data?

I was recently working on some classification problem where decision trees performed better than neural networks. I had tried various combinations with neural networks altering the number of neurons / ...
Suhail Gupta's user avatar
11 votes
3 answers
3k views

Can regression trees predict continuously?

Suppose I have a smooth function like $f(x, y) = x^2+y^2$. I have a training set $D \subsetneq \{((x, y), f(x,y)) | (x,y) \in \mathbb{R}^2\}$ and, of course, I don't know $f$ although I can evaluate $...
Martin Thoma's user avatar
11 votes
1 answer
30k views

Decision tree, how to understand or calculate the probability/confidence of prediction result

For example, a drug prediction problem using a decision tree. I trained the decision tree model and would like to predict using new data. For example: ...
GoingMyWay's user avatar
10 votes
3 answers
5k views

LightGBM - Why Exclusive Feature Bundling (EFB)?

I'm currently studying GBDT and started reading LightGBM's research paper. In section 4. they explain the Exclusive Feature Bundling algorithm, which aims at reducing the number of features by ...
Tom's user avatar
  • 103
9 votes
2 answers
21k views

Why don't tree ensembles require one-hot-encoding?

I know that models such as random forest and boosted trees don't require one-hot encoding for predictor levels, but I don't really get why. If the tree is making a split in the feature space, then isn'...
moefasa's user avatar
  • 93
8 votes
2 answers
26k views

How to interpret a decision tree correctly?

I'm trying to work out if I'm correctly interpreting a decision tree found online. The dependent variable of this decision tree is Credit Rating which has two classes, Bad or Good. The root of this ...
DataD's user avatar
  • 83
8 votes
1 answer
198 views

Which ML approach to choose for the game AI when rewards are delayed?

Question: Which Machine Learning approach should I choose for the AI of my computer game, where the actions of the AI do not lead to immediate rewards, but delayed rewards instead? About me: I am a ...
Logende's user avatar
  • 61
6 votes
1 answer
8k views

Why we use information gain over accuracy as splitting criterion in decision tree?

In decision tree classifier most of the algorithms use Information gain as spiting criterion. We select the feature with maximum information gain to split on. I think that using accuracy instead of ...
Anwar Shaikh's user avatar
5 votes
2 answers
740 views

Sklearn: applying cost complexity pruning along with pipeline

I have a data set with categorical variables. I have defined a decision tree algorithm and transformed these columns to numerical equivalent using one hot encoding functionality in sklearn: Create ...
codeman340's user avatar
5 votes
2 answers
672 views

Loss for ordered multi class data in classification

Assume data which is labeled $y_i \in \left\{ 1, 2, 3, \ldots, 9, 10\right\}$. Assume the labels are ordered, namely, given $y_i = 10$ to estimate $\hat{y}_{i} = 1$ is much worse than $\hat{y}_{i} = ...
Eric Johnson's user avatar
4 votes
1 answer
287 views

Numeric variables in Decision trees

If we have numeric variable, decision trees will use < and > comparisons as splitting criteria. Lets consider this case : ...
Venkatesh Gandi's user avatar
4 votes
2 answers
8k views

Ordinal feature in decision tree

I am curious if ordinal features are treated differently from categorical features in decision tree, I am interested in both cases where target is categorical or continuous. If there is a difference, ...
user1700890's user avatar
4 votes
2 answers
4k views

Fix first two levels of decision tree?

I am trying to build a regression tree with 70 attributes where the business team wants to fix the first two levels namely country and product type. To achieve this, I have two proposals: Build a ...
Aravind's user avatar
  • 41
4 votes
1 answer
3k views

selecting variable randomly at each node in a tree in Random Forest

In Random Forest method, for each tree we randomly select a set of variables (features) of fixed size. But once this set is frozen for that particular tree, does the tree behave like a regular ...
Dhiraj's user avatar
  • 233
4 votes
1 answer
4k views

Credit scoring using scorecardpy with XGBoost

I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
Minila S's user avatar
4 votes
2 answers
2k views

Ordinal Attributes in a Decision Tree

I'm reading the book Introduction to Data Mining by Tan, Steinbeck, and Kumar. In the chapter on Decision Trees, when talking about the "Methods for Expressing Attribute Test Conditions" the book says ...
Koinos's user avatar
  • 141
3 votes
1 answer
1k views

How is the 'feature_importance_' value calculated in sklearn random forest regressor?

I have 9000 sample, with five features, and one output variable (all are numerical, continuous values). I used random forest regression method using scikit modules. ...
Suvardhan Jonnalagadda's user avatar
3 votes
1 answer
3k views

Extract the "path" of a data point through a decision tree in sklearn

I'm working with decision trees in python's scikit learn. Unlike many use cases for this, I'm not so much interested in the accuracy of the classifier at this point so much as I am extracting the ...
Kyle.'s user avatar
  • 1,483
3 votes
3 answers
143 views

Better approach to assign values to determine potential fake sentences

I am trying to assign different values for each sentences based on information about the presence of hashtags, upper case letters/words (e.g. HATE) and some others. I created a data frame which ...
Math's user avatar
  • 161
3 votes
1 answer
93 views

Can I create random forest with RandomForestClassifier which will consist the same trees?

Based on answers to this question, I should be able to build a random forest with all the same trees by using ...
Damian Melniczuk's user avatar
2 votes
2 answers
647 views

What can weka do that python and sklearn can't?

I would like to build a variety of classification and regression decision trees. My use case focuses on extraction and communication of decision rules. Previously weka was used in my organisation for ...
Matthew Stewart's user avatar
2 votes
1 answer
187 views

How does bagging help reduce the variance

I learned that bagging helps reduce variance by averaging but I couldn't understand this. Can someone explain this intuitively?
Bhuwan Bhatt's user avatar
2 votes
2 answers
345 views

How Decision Tree Classifier works? [closed]

In particular i am using SKLearn with class DecisionTreeClassifier. I would really like to understand how the tree build itself ...
baltiturg's user avatar
  • 143
2 votes
4 answers
138 views

Predicting Disease Drugs

I have a dataset in the format: ...
Atom Store's user avatar
2 votes
4 answers
7k views

is it possible to output more than 2 nodes away from a node in a decision tree? if yes, how to do that with sklearn?

usually a decision tree has one root node, some nodes, and some leaves. lots tutorial illustrate this as something like binary tree. is it possible more than 2 nodes away from a node in a decision ...
JJJohn's user avatar
  • 623
2 votes
0 answers
626 views

How to come up with the splitting point in a decision tree? [duplicate]

I read https://www.researchgate.net/post/How_to_compute_impurity_using_Gini_Index I understand why choosing smallest gini index, but how do I come up with different candidate splits in the first ...
Leo Jiang's user avatar