# Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

619 questions
Filter by
Sorted by
Tagged with
122k views

### When should I use Gini Impurity as opposed to Information Gain (Entropy)?

Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)? Which metric is better to use in different scenarios while using decision trees?
118k views

### strings as features in decision tree/random forest

I am doing some problems on an application of decision tree/random forest. I am trying to fit a problem which has numbers as well as strings (such as country name) as features. Now the library, scikit-...
34k views

### Why do we need XGBoost and Random Forest?

I wasn't clear on couple of concepts: XGBoost converts weak learners to strong learners. What's the advantage of doing this ? Combining many weak learners instead of just using a single tree ? ...
41k views

### Are decision tree algorithms linear or nonlinear

Recently a friend of mine was asked whether decision tree algorithms are linear or nonlinear algorithms in an interview. I tried to look for answers to this question but couldn't find any satisfactory ...
73k views

### How to predict probabilities in xgboost?

The below predict function is giving -ve values as well so it cannot be probabilities. ...
24k views

### How is a splitting point chosen for continuous variables in decision trees?

I have two questions related to decision trees: If we have a continuous attribute, how do we choose the splitting value? Example: Age=(20,29,50,40....) Imagine that we have a continuous attribute $f$...
30k views

### XGBRegressor vs. xgboost.train huge speed difference?

If I train my model using the following code: ...
45k views

### Should I use a decision tree or logistic regression for classification?

I am working on a classification problem. I have a dataset containing equal numbers of categorical variables and continuous variables. How do I decide which technique to use, between a decision tree ...
35k views

### Decision tree vs. KNN

In which cases is it better to use a Decision tree and other cases a KNN? Why use one of them in certain cases? And the other in different cases? (By looking at its functionality, not at the ...
11k views

### Is it necessary to normalise data for XGBoost?

MinMaxScaler in scikit_learn is used for data normalization (a.k.a feature scaling). Data normalisation is not necessary for decision trees. Since XGBoost is based on decision trees, is it necessary ...
10k views

### Decision trees: leaf-wise (best-first) and level-wise tree traverse

Issue 1: I am confused by the description of LightGBM regarding the way the tree is expanded. They state: Most decision tree learning algorithms grow tree by level (depth)-wise, like the ...
403 views

### Can gradient boosted trees fit any function?

For neural networks we have the universal approximation theorem which states that neural networks can approximate any continuous function on a compact subset of $R^n$. Is there a similar result for ...
50k views

### How to make a decision tree with both continuous and categorical variables in the dataset?

Let's say I have 3 categorical and 2 continuous attributes in a dataset. How do I build a decision tree using these 5 variables? Edit: For categorical variables, it is easy to say that we will split ...
30k views

### How to normalize data for Neural Network and Decision Forest

I have a data set with 20000 samples, each has 12 different features. Each sample is either in category 0 or 1. I want to train a neural network and a decision forest to categorize the samples so that ...
63k views

### When to choose linear regression or Decision Tree or Random Forest regression? [closed]

I am working on a project and I am having difficulty in deciding which algorithm to choose for regression. I want to know under what conditions should one choose a <...
18k views

### Unbalanced classes — How to minimize false negatives?

I have a dataset that has a binary class attribute. There are 623 instances with class +1 (cancer positive) and 101,671 instances with class -1 (cancer negative). I've tried various algorithms (Naive ...
17k views

### Interpreting Decision Tree in context of feature importances

I'm trying to understand how to fully understand the decision process of a decision tree classification model built with sklearn. The 2 main aspect I'm looking at are a graphviz representation of the ...
6k views

### Catboost Categorical Features Handling Options (CTR settings)?

I am working with a dataset with large number of categorical features (>80%) predicting a continuous target variable (i.e. Regression). I have been reading quite a bit about ways to handle categorical ...
3k views

### Is max_depth in scikit the equivalent of pruning in decision trees?

I was analyzing the classifier created using a decision tree. There is a tuning parameter called max_depth in scikit's decision tree. Is this equivalent of pruning a decision tree? If not, how could I ...
4k views

### Why neural networks do not perform well on structured data?

I was recently working on some classification problem where decision trees performed better than neural networks. I had tried various combinations with neural networks altering the number of neurons / ...
22k views

### XGBoost for binary classification: choosing the right threshold

I am working on a highly-imbalanced binary-labeled dataset, where number of true labels is just 7% from the whole dataset. But some combination of features could yield higher than average number of ...
2k views

6k views

### Does feature selections matter to Decision Tree algorithms?

Background: Currently I'm working on my thesis project, which is to build Tree-based ensemble methods for classification on a large data set. Before I started with modeling, I've spent a large amount ...
5k views

### What feature engineering is necessary with tree based algorithms?

I understand data hygiene, which is probably the most basic feature engineering. That is making sure all your data is properly loaded, making sure N/As are treated ...
5k views

### Why decision tree needs categorical variable to be encoded?

As per my intuition, decision trees should work better with categorical variables than with continuous variables. If this is the case, why is encoding needed on categorical variables? Can someone give ...
10k views

### How to get a confidence score for predictions?

In a regression problem, is it possible to calculate a confidence/reliability score for a certain prediction given models like XGBoost or Neural Networks?
2k views

### How to (better) discretize continuous data in decision trees?

Standard decision tree algorithms, such as ID3 and C4.5, have a brute force approach for choosing the cut point in a continuous feature. Every single value is tested as a possible cut point. (By ...
3k views

### Minimum number of trees for Random Forest classifier

I am searching for a theoretical or experimental estimation of the lower bound for the number of trees in a Random Forest classifier. I usually test different combinations and select the one that (...
10k views

### Information Gain in R

I found packages being used to calculating "Information Gain" for selecting main attributes in C4.5 Decision Tree and I tried using them to calculating "Information Gain". But the results of ...
142 views

### Which ML approach to choose for the game AI when rewards are delayed?

Question: Which Machine Learning approach should I choose for the AI of my computer game, where the actions of the AI do not lead to immediate rewards, but delayed rewards instead? About me: I am a ...
11k views

### Multicollinearity in Decision Tree

Can anybody please explain the affect of multicollinearity on Decision Tree algorithms (Classification and regression). I have done some searching but was not able to find the right answer as some say ...
11k views

### How to preprocess different kinds of data (continuous, discrete, categorical) before Decision Tree learning

I want to use some Decision Tree learning, such as the Random Forest classifier. I have data of different types: continuous, discrete and categorical. How do I have to preprocess data in order to ...
14k views

### Why don't tree ensembles require one-hot-encoding?

I know that models such as random forest and boosted trees don't require one-hot encoding for predictor levels, but I don't really get why. If the tree is making a split in the feature space, then isn'...
22k views

### How to prevent/tell if Decision Tree is overfitting?

In SKLearn's documentation on Decision Trees, they say we should pay special attention not to overfit the tree. How can we do this? I am aware that using random forests may prevent it, but how do I ...
21k views

### How to interpret a decision tree correctly?

I'm trying to work out if I'm correctly interpreting a decision tree found online. The dependent variable of this decision tree is Credit Rating which has two classes, Bad or Good. The root of this ...
70 views

### How to decide the shape of input features, when each data file is of different length?

To help me understand the benefits and shortcomings of decision trees, KNN, Neural Networks, ...
725 views

### Is there any difference between a weak learner and a weak classifier?

While reading about decision tree ensembles Gradient Boosting, AdaBoost etc. I have found the following two concepts weak learner and weak classifier. Are they the same? If there is any difference ...