Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
1
vote
1answer
14 views

Reasons for a model predicting probability of being class 1 at x value

All, This is a general question. I have a binary classification which predicts if someone is rich or not. I had a question from someone asking that if the probability someone is rich is 0.6 and ...
0
votes
0answers
10 views

Looking for an algorithm to perform classification on multivariate grouped time series

I will be grateful for any help. I have multivariate time series, where every one of them has an unique ID. Also, there is a variable giving information about the trend type of the ID from a point of ...
5
votes
1answer
66 views

Multi-target regression tree with additional constraint

I have a regression problem where I need to predict three dependent variables ($y$) based on a set of independent variables ($x$): $$ (y_1,y_2,y_3) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \...
1
vote
1answer
59 views

Decision Tree taking too long to execute

I am training a Decision Tree Regressor on a relatively small data. The dimensions of my train and test sets are (34164, 10) and (8514, 10). Here is the relevant code: ...
0
votes
0answers
18 views

Handling repeating data from different individuals

I have a dataset that has some unique values but also includes information from multiple individuals that are repeating, meaning they are describing the same attributes and can have the same or ...
2
votes
2answers
47 views

Is it possible to do hard-coded decision tree on some variables and random forest / something on the remaining ones?

Is it possible to do hard-coded decision tree on some variables and random forest / something on the remaining ones? The situation seems that for some variables it's possible to draw strong empirical ...
0
votes
0answers
9 views

How to apply post pruning methods to ID3 decision tree

I am developing an ID3 decision tree implementation that feature post-pruning and classification. The program below constructs the decision tree. ...
0
votes
0answers
7 views

Decision Trees and SHAP Values

I've recently been using some (optimal) decision trees methods in R, such as 'evtree' and 'iai.' Both of these provide really nice interpretable plots. And out of the 12 covariates I have in my model, ...
99
votes
9answers
125k views

When should I use Gini Impurity as opposed to Information Gain (Entropy)?

Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)? Which metric is better to use in different scenarios while using decision trees?
0
votes
2answers
36 views

How to implement ID3

I'm trying to follow the suggested outline form implementing ID3 ...
0
votes
0answers
8 views

How to extract rules for decision tree from ID3 classification

I am working on a program to implement the ID3 algorithm. The program takes in user input for setting a threshold and creating a decision tree. ...
0
votes
3answers
19 views

Creating numeric word representation of input sentences resulting in MemoryError

I am trying to use CountVectorizer to obtain word numerical word representation of data which is essentialy list of 160000 English sentences: ...
1
vote
0answers
38 views

Theoretical maximum depth of a decision tree

During my machine learning labwork, I was trying to fit a decision tree to the IRIS dataset (150 samples, 4 features). The maximum theoretical depth my tree can reach which is, for my understanding, ...
1
vote
3answers
297 views

Advantages and disadvantages of using classification tree

I was working on a project and was trying to validate my decisions. I wondered why would I want to use a decision tree over more powerful algorithms like random forest or Gradient boosting machine ...
0
votes
1answer
27 views

Catboost not able to handle a very simple dataset?

This is a post from a newbie and so might be a really poor question based on lack of knowledge. Thank you kindly! I'm using Catboost, which seems excellent, to fit a trivial dataset. The results are ...
1
vote
1answer
89 views

Upper bound on size of sample set for decision trees

Say I have an instance space with 4 features and I know that a decision tree with 8 nodes can represent the target function I want to learn. I want to give an upper bound on the size of the sample set ...
2
votes
4answers
95 views
0
votes
0answers
15 views

Could gini impurity rise as we go through decision tree?

I have a DecisionTreeClassifier built with sklearn (criterion="gini"), for which I need to explain how each particular ...
0
votes
1answer
55 views

How can we shorten our questionnaire to only ask the most informative question at each point?

Our product has an onboarding questionnaire which asks the same 58 questions (with numeric answers) to every new user. That’s a lot of questions, so we’d love to reduce the number of questions we ask ...
-1
votes
0answers
31 views

How to extract rules from training set to run against test set for ID3 algorithm

I am working on a data mining project. The goal of this project is to implement an ID3 classifier and Naive Bayes Classifer. I have to sets of data for ID3 the following test sets are provided A2 ...
3
votes
3answers
162 views

CART algorithm (Classification and regression trees) question

We fit a full classification tree model $T_k$ of given depth $k$ to data using the CART algorithm, and prune the tree by finding $E(k, \alpha) = min_{T\subset Tk} Err(T) + \alpha |T|$. Here, $Err(T)$ ...
3
votes
0answers
49 views

Non-greedy decision tree / random forest implementation(s) in Python

The standard random forest is trained using a greedy approach for computational feasibility. However, there are a number of alternative methods such as "lookahead" or using bilevel ...
1
vote
1answer
29 views

Building a linear regression model for every combination vs only one Machine Learning model

So my question is more on the conceptual side. Given a dataset, I want to predict a given continuous variable Y. Now, there are 3 features, 2 categorical and one numerical (integer only). I know that ...
1
vote
1answer
50 views
1
vote
0answers
17 views

How to determine which classes are easier to predict with a decision tree?

So, I'm trying to work with decision trees on Iris dataset. I've noticed by trying out different parameter (max_depth, leaves etc) that some of the classes are easier to predict (most of the trees ...
1
vote
1answer
11 views

Encoding distance variable that is continuous until out-of-range

I have a varaible distance which is continous until a "hard stop" at which we stop measuring the distance itself and just label the distance as "out ...
0
votes
1answer
30 views

Feature importance by random forest and boosting tree when two features are heavy correlated [closed]

I have asked this question here but seems no one is interested in it. Here is my understanding, pls correct me if there is any misunderstanding: Tree models is used ...
1
vote
1answer
18 views

If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree

I have asked this question here, but seems no one was interested in it: https://stats.stackexchange.com/questions/550994/if-a-feature-has-already-split-will-it-hardly-be-selected-to-split-again-in-the ...
0
votes
0answers
10 views

Class Weight in sklearn DecisionTreeClassifier impact during prediction

I understand that class weights are used during splitting to weigh whatever metric in the children of the split. However I cannot find anywhere whether class weights also impact prediction or are ...
1
vote
0answers
20 views

How are regression trees fitted in gradient boosting for classification?

What I understood is that even gradient boosting for binary classification uses regression trees. The first value we calculate is constant = log(odds). For the rest of the trees, we try to fit ...
1
vote
0answers
11 views
4
votes
1answer
89 views

How is bayesian risk computed to prune decision trees?

I've been trying to follow this paper on Bayesian Risk Pruning. I'm not very familiar with this type of pruning, but I'm wondering a few things: (1) The paper describes risk-rates to be defined per ...
4
votes
1answer
38 views

Output value of a gradient boosting decision tree node that has just a single example in it

The general gradient boosting algorithm for tree-based classifiers is as follows: Input: training set $\{(x_{i},y_{i})\}_{i=1}^{n}$, a differentiable loss function $L(y,F(x))$, and a number of ...
3
votes
1answer
29 views

classification balanced target y [0,1] but imbalanced feature x [many 0 , few 1s] , maximize precision

I have a simple dataset with balanced target y (0 or 1) ,and imbalanced feature (many 0 , few 1's) I aim to get high precision (don't care about recall) I can get high precision of 0.53 if I just ...
5
votes
1answer
1k views

Is there a real C4.5 implementation in Python ? (handling missing value)

To my understanding, C4.5 comes with 4 improvements compared to ID3: Handling missing values in both training data and "test" data, Handling continuous data Handling costs on attributes. The pruning ...
2
votes
1answer
76 views

Decision tree Why is Gini index only used for binary choices?

I would like to understand why "Gini index operates on the categorical target variables in terms of “success” or “failure” and performs only binary split" ? Why it would not be possible to ...
2
votes
2answers
216 views

Decision Tree Induction using Information Gain and Entropy

I’m trying to build a decision tree algorithm, but I think I misinterpreted how information gain works. Let’s say we have a balanced classification problem. So, the initial entropy should equal 1. ...
2
votes
1answer
48 views

How to model a supervised recommender system with varying data

Suppose there are 2000 movies and a company wants to recommend some movies (for example, at most 5 movies) to each visitor. The objective is to learn how to predict which movie will be selected if a ...
1
vote
1answer
43 views

What does "S" in Shannon's entropy stands for?

I see many machine learning texts using the following notation to represent Shannon's entropy in classification/supervised learning contexts: $$ H(S) = \sum_{i \in Y}p_i \log(p_i) $$ Where $p_i$ is ...
0
votes
1answer
93 views

Syntax error but nothing appears to be wrong?

I'm trying to create a decision tree, and everything appeared to be going smoothly until I encountered this syntax error. I don't see what the issue is? ...
1
vote
0answers
19 views

Is there a "tree-based-correlation" for tree-based algorithms?

Although correlated features are not a big issue when training tree-based models, they spoil model explainability. When several features correlate, sometimes they may be picked at random. Then their ...
2
votes
1answer
70 views

Machine Learning - Same impurity values

Imagine this example: We see that the attributes color and points have the same value. What attribute should we choose for the ...
2
votes
2answers
227 views

How to decode encoded labels in Decision tree classifier

I have some dataset with procurements of organization where actually i'm working. The aim is to find most important features that describe why some processes of purchases is succesful, and why not ...
0
votes
1answer
112 views

NN Making Poor Averaging Fit whilst LGBM Regressor Fits Perfectly

I have a simple toy dataset for which the features have been encoded using a Encoder-Decoder NN. I am using the hidden feature vector from the Encoder as the X input for training a 1-step lookahead ...
1
vote
1answer
28 views

How different classifiers would perform on a particular data set

I am reading through and learning how different ML methods work on different types of data, but I have faced a data set that I am not sure how ML methods, such as decision tree, Naive Bayes, and KNN, ...
1
vote
2answers
22 views

Multiple models have extreme differences during evaluation

My dataset has about 100k entries, 6 features, and the label is simple binary classification (about 65% zeros, 35% ones). When I train my dataset on different models: random forest, decision tree, ...
0
votes
1answer
115 views

Proof that Gini Impurity in a Decision Tree is Monotone Decreasing?

I asked this in a reply to an answer to another of my questions; but I think this merits its own question since I couldn't find an answer, and it's a pretty interesting question on its own. Suppose we ...
1
vote
2answers
170 views

separate decision tree for categorical feature values

Given either, different decision trees each based on a particular feature value (like separate models for each male and female) or a single decision tree, should both give the same result?
1
vote
1answer
164 views

Model retraining

I trained my model with RandomForestRegressor(), but now my training data is updated continuously. So I have to train my model with all the train data set i.e past ...

1
2 3 4 5
14