Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

54 questions
Filter by
Sorted by
Tagged with
6k views

Why decision tree needs categorical variable to be encoded?

As per my intuition, decision trees should work better with categorical variables than with continuous variables. If this is the case, why is encoding needed on categorical variables? Can someone give ...
26k views

How is a splitting point chosen for continuous variables in decision trees?

I have two questions related to decision trees: If we have a continuous attribute, how do we choose the splitting value? Example: Age=(20,29,50,40....) Imagine that we have a continuous attribute $f$...
712 views

Is Label Encoding with arbitrary numbers ever useful at all?

From what I read online, there seems to be some confusion regarding the taxonomy and the terms used, so to avoid misunderstanding I'm going to define them here: Label Encoding - encoding a nominal ...
243 views

Can a decision in a node of a decision tree be based on comparison between 2 columns of the dataset?

Assume the features in the dataframe are columns - A,B,C and my target is Y Can my decision tree have a decision node which looks for say, ...
3k views

When does decision tree perform better than the neural network?

I was experimenting with different modelling methods including KNN, Decision Trees, Neural Networks and SVN and trying to fit my data to see which works the best. To my surprise, the decision tree ...
125k views

When should I use Gini Impurity as opposed to Information Gain (Entropy)?

Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)? Which metric is better to use in different scenarios while using decision trees?
124k views

strings as features in decision tree/random forest

I am doing some problems on an application of decision tree/random forest. I am trying to fit a problem which has numbers as well as strings (such as country name) as features. Now the library, scikit-...
58k views

How to make a decision tree with both continuous and categorical variables in the dataset?

Let's say I have 3 categorical and 2 continuous attributes in a dataset. How do I build a decision tree using these 5 variables? Edit: For categorical variables, it is easy to say that we will split ...
30k views

How to normalize data for Neural Network and Decision Forest

I have a data set with 20000 samples, each has 12 different features. Each sample is either in category 0 or 1. I want to train a neural network and a decision forest to categorize the samples so that ...
13k views

Multicollinearity in Decision Tree

Can anybody please explain the affect of multicollinearity on Decision Tree algorithms (Classification and regression). I have done some searching but was not able to find the right answer as some say ...
3k views

Can a decision tree learn to solve a xOR problem?

I have read online that decision trees can solve xOR type problems, as shown in images (xOR problem: 1) and (Possible solution as decision tree: 2). My question is how can a decision tree learn to ...
3k views

Is max_depth in scikit the equivalent of pruning in decision trees?

I was analyzing the classifier created using a decision tree. There is a tuning parameter called max_depth in scikit's decision tree. Is this equivalent of pruning a decision tree? If not, how could I ...
84 views

Make a random forest estimator the exact same of a decision tree

The idea is to make one of the trees of a Random Forest, to be built exactly equal to a Decision Tree. First, we load all libraries, fit a decision tree and plot it. ...
956 views

Decision tree not using all features from training dataset

I have built CART model using sklearn. I'm having total 6 features in training dataset and passing all of them in fit function. I've tested both criteria Gini and entropy. But whenever I plot tree ...
35k views

Why do we need XGBoost and Random Forest?

I wasn't clear on couple of concepts: XGBoost converts weak learners to strong learners. What's the advantage of doing this ? Combining many weak learners instead of just using a single tree ? ...
46k views

Are decision tree algorithms linear or nonlinear

Recently a friend of mine was asked whether decision tree algorithms are linear or nonlinear algorithms in an interview. I tried to look for answers to this question but couldn't find any satisfactory ...
16k views

Why don't tree ensembles require one-hot-encoding?

I know that models such as random forest and boosted trees don't require one-hot encoding for predictor levels, but I don't really get why. If the tree is making a split in the feature space, then isn'...
31k views

XGBRegressor vs. xgboost.train huge speed difference?

If I train my model using the following code: ...
5k views

Why neural networks do not perform well on structured data?

I was recently working on some classification problem where decision trees performed better than neural networks. I had tried various combinations with neural networks altering the number of neurons / ...
18k views

Interpreting Decision Tree in context of feature importances

I'm trying to understand how to fully understand the decision process of a decision tree classification model built with sklearn. The 2 main aspect I'm looking at are a graphviz representation of the ...
23k views

Decision tree, how to understand or calculate the probability/confidence of prediction result

For example, a drug prediction problem using a decision tree. I trained the decision tree model and would like to predict using new data. For example: ...
6k views

Why we use information gain over accuracy as splitting criterion in decision tree?

In decision tree classifier most of the algorithms use Information gain as spiting criterion. We select the feature with maximum information gain to split on. I think that using accuracy instead of ...
19k views

Unbalanced classes -- How to minimize false negatives?

I have a dataset that has a binary class attribute. There are 623 instances with class +1 (cancer positive) and 101,671 instances with class -1 (cancer negative). I've tried various algorithms (Naive ...
15k views

Is it necessary to normalize data for XGBoost?

MinMaxScaler() in scikit-learn is used for data normalization (a.k.a feature scaling). Data normalization is not necessary for ...
2k views

LightGBM - Why Exclusive Feature Bundling (EFB)?

I'm currently studying GBDT and started reading LightGBM's research paper. In section 4. they explain the Exclusive Feature Bundling algorithm, which aims at reducing the number of features by ...
102 views

Better approach to assign values to determine potential fake sentences

I am trying to assign different values for each sentences based on information about the presence of hashtags, upper case letters/words (e.g. HATE) and some others. I created a data frame which ...
151 views

Which ML approach to choose for the game AI when rewards are delayed?

Question: Which Machine Learning approach should I choose for the AI of my computer game, where the actions of the AI do not lead to immediate rewards, but delayed rewards instead? About me: I am a ...
22k views

How to interpret a decision tree correctly?

I'm trying to work out if I'm correctly interpreting a decision tree found online. The dependent variable of this decision tree is Credit Rating which has two classes, Bad or Good. The root of this ...
777 views

How is the 'feature_importance_' value calculated in sklearn random forest regressor?

I have 9000 sample, with five features, and one output variable (all are numerical, continuous values). I used random forest regression method using scikit modules. ...
2k views

Can regression trees predict continuously?

Suppose I have a smooth function like $f(x, y) = x^2+y^2$. I have a training set $D \subsetneq \{((x, y), f(x,y)) | (x,y) \in \mathbb{R}^2\}$ and, of course, I don't know $f$ although I can evaluate \$...
3k views

selecting variable randomly at each node in a tree in Random Forest

In Random Forest method, for each tree we randomly select a set of variables (features) of fixed size. But once this set is frozen for that particular tree, does the tree behave like a regular ...
3k views

Extract the "path" of a data point through a decision tree in sklearn

I'm working with decision trees in python's scikit learn. Unlike many use cases for this, I'm not so much interested in the accuracy of the classifier at this point so much as I am extracting the ...
415 views

Can I force DecisionTreeClassifier to use integer conditions when the variable is integer?

I'm trying to visualize a decision tree in python for the purpose of explainability. I noticed that a condition like "NumGoals >= 1.23" could be quite vague for the user and I would much rather to see ...
291 views

Sklearn: applying cost complexity pruning along with pipeline

I have a data set with categorical variables. I have defined a decision tree algorithm and transformed these columns to numerical equivalent using one hot encoding functionality in sklearn: Create ...
393 views

Decision Trees Should We Discard Low Importance Features?

I just started to work with feature selection. Let's say I have a decision tree model. I get its feature importances by tree.feature_importances_. In my model out ...
3k views

Fix first two levels of decision tree?

I am trying to build a regression tree with 70 attributes where the business team wants to fix the first two levels namely country and product type. To achieve this, I have two proposals: Build a ...
6k views

Ordinal feature in decision tree

I am curious if ordinal features are treated differently from categorical features in decision tree, I am interested in both cases where target is categorical or continuous. If there is a difference, ...
83 views

Can I create random forest with RandomForestClassifier which will consist the same trees?

Based on answers to this question, I should be able to build a random forest with all the same trees by using ...
61 views

How does bagging help reduce the variance

I learned that bagging helps reduce variance by averaging but I couldn't understand this. Can someone explain this intuitively?
3k views

is it possible to output more than 2 nodes away from a node in a decision tree? if yes, how to do that with sklearn?

usually a decision tree has one root node, some nodes, and some leaves. lots tutorial illustrate this as something like binary tree. is it possible more than 2 nodes away from a node in a decision ...
303 views

How does class_weight work in Decision Tree?

I am interested in Cost-Sensitive learning. And I am trying to understand how class_weight in DecisionTree works in terms of math. I read a lot of articles that ...
8k views

Why max_features=n_features does not make the Random Forest independent of number of trees?

Consider the following simple classification problem (Python, scikit-learn) ...
145 views

Numeric variables in Decision trees

If we have numeric variable, decision trees will use < and > comparisons as splitting criteria. Lets consider this case : ...
1k views

Ordinal Attributes in a Decision Tree

I'm reading the book Introduction to Data Mining by Tan, Steinbeck, and Kumar. In the chapter on Decision Trees, when talking about the "Methods for Expressing Attribute Test Conditions" the book says ...
99 views

Predicting Disease Drugs

I have a dataset in the format: ...
580 views

How to come up with the splitting point in a decision tree? [duplicate]

I read https://www.researchgate.net/post/How_to_compute_impurity_using_Gini_Index I understand why choosing smallest gini index, but how do I come up with different candidate splits in the first ...
80 views

Decision Tree taking too long to execute

I am training a Decision Tree Regressor on a relatively small data. The dimensions of my train and test sets are (34164, 10) and (8514, 10). Here is the relevant code: ...
323 views

XGBoost - Imputing Vs keeping NaN

What is the benefit of imputing numerical or categorical features when using DT methods such as XGBoost that can handle missing values? This question is mainly for when the values are missing not at ...