Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
1
vote
1answer
16 views

Reasons for a model predicting probability of being class 1 at x value

All, This is a general question. I have a binary classification which predicts if someone is rich or not. I had a question from someone asking that if the probability someone is rich is 0.6 and ...
0
votes
0answers
10 views

Looking for an algorithm to perform classification on multivariate grouped time series

I will be grateful for any help. I have multivariate time series, where every one of them has an unique ID. Also, there is a variable giving information about the trend type of the ID from a point of ...
0
votes
0answers
18 views

Handling repeating data from different individuals

I have a dataset that has some unique values but also includes information from multiple individuals that are repeating, meaning they are describing the same attributes and can have the same or ...
0
votes
0answers
9 views

How to apply post pruning methods to ID3 decision tree

I am developing an ID3 decision tree implementation that feature post-pruning and classification. The program below constructs the decision tree. ...
0
votes
0answers
7 views

Decision Trees and SHAP Values

I've recently been using some (optimal) decision trees methods in R, such as 'evtree' and 'iai.' Both of these provide really nice interpretable plots. And out of the 12 covariates I have in my model, ...
0
votes
0answers
9 views

How to extract rules for decision tree from ID3 classification

I am working on a program to implement the ID3 algorithm. The program takes in user input for setting a threshold and creating a decision tree. ...
0
votes
2answers
36 views

How to implement ID3

I'm trying to follow the suggested outline form implementing ID3 ...
0
votes
3answers
19 views

Creating numeric word representation of input sentences resulting in MemoryError

I am trying to use CountVectorizer to obtain word numerical word representation of data which is essentialy list of 160000 English sentences: ...
1
vote
0answers
38 views

Theoretical maximum depth of a decision tree

During my machine learning labwork, I was trying to fit a decision tree to the IRIS dataset (150 samples, 4 features). The maximum theoretical depth my tree can reach which is, for my understanding, ...
0
votes
1answer
27 views

Catboost not able to handle a very simple dataset?

This is a post from a newbie and so might be a really poor question based on lack of knowledge. Thank you kindly! I'm using Catboost, which seems excellent, to fit a trivial dataset. The results are ...
0
votes
0answers
15 views

Could gini impurity rise as we go through decision tree?

I have a DecisionTreeClassifier built with sklearn (criterion="gini"), for which I need to explain how each particular ...
-1
votes
0answers
31 views

How to extract rules from training set to run against test set for ID3 algorithm

I am working on a data mining project. The goal of this project is to implement an ID3 classifier and Naive Bayes Classifer. I have to sets of data for ID3 the following test sets are provided A2 ...
1
vote
1answer
29 views

Building a linear regression model for every combination vs only one Machine Learning model

So my question is more on the conceptual side. Given a dataset, I want to predict a given continuous variable Y. Now, there are 3 features, 2 categorical and one numerical (integer only). I know that ...
1
vote
0answers
17 views

How to determine which classes are easier to predict with a decision tree?

So, I'm trying to work with decision trees on Iris dataset. I've noticed by trying out different parameter (max_depth, leaves etc) that some of the classes are easier to predict (most of the trees ...
1
vote
1answer
11 views

Encoding distance variable that is continuous until out-of-range

I have a varaible distance which is continous until a "hard stop" at which we stop measuring the distance itself and just label the distance as "out ...
2
votes
4answers
95 views

Predicting Disease Drugs

I have a dataset in the format: ...
0
votes
1answer
30 views

Feature importance by random forest and boosting tree when two features are heavy correlated [closed]

I have asked this question here but seems no one is interested in it. Here is my understanding, pls correct me if there is any misunderstanding: Tree models is used ...
1
vote
1answer
18 views

If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree

I have asked this question here, but seems no one was interested in it: https://stats.stackexchange.com/questions/550994/if-a-feature-has-already-split-will-it-hardly-be-selected-to-split-again-in-the ...
0
votes
0answers
10 views

Class Weight in sklearn DecisionTreeClassifier impact during prediction

I understand that class weights are used during splitting to weigh whatever metric in the children of the split. However I cannot find anywhere whether class weights also impact prediction or are ...
1
vote
0answers
11 views

Difference between rpart models, one with information split the other with rpart.control

What is the difference between these two models? ...
3
votes
1answer
29 views

classification balanced target y [0,1] but imbalanced feature x [many 0 , few 1s] , maximize precision

I have a simple dataset with balanced target y (0 or 1) ,and imbalanced feature (many 0 , few 1's) I aim to get high precision (don't care about recall) I can get high precision of 0.53 if I just ...
2
votes
1answer
76 views

Decision tree Why is Gini index only used for binary choices?

I would like to understand why "Gini index operates on the categorical target variables in terms of “success” or “failure” and performs only binary split" ? Why it would not be possible to ...
1
vote
1answer
43 views

What does "S" in Shannon's entropy stands for?

I see many machine learning texts using the following notation to represent Shannon's entropy in classification/supervised learning contexts: $$ H(S) = \sum_{i \in Y}p_i \log(p_i) $$ Where $p_i$ is ...
1
vote
0answers
19 views

Is there a "tree-based-correlation" for tree-based algorithms?

Although correlated features are not a big issue when training tree-based models, they spoil model explainability. When several features correlate, sometimes they may be picked at random. Then their ...
1
vote
1answer
28 views

How different classifiers would perform on a particular data set

I am reading through and learning how different ML methods work on different types of data, but I have faced a data set that I am not sure how ML methods, such as decision tree, Naive Bayes, and KNN, ...
0
votes
2answers
32 views
1
vote
0answers
13 views

What kind of model to use to find drivers when data is aggregated and not on user level?

I have a website and have info from Google Analytics. So I can see the following "features": page url country device category (phone, desktop, etc.) Number of sessions Number of users: ...
1
vote
2answers
22 views

Multiple models have extreme differences during evaluation

My dataset has about 100k entries, 6 features, and the label is simple binary classification (about 65% zeros, 35% ones). When I train my dataset on different models: random forest, decision tree, ...
3
votes
0answers
49 views

Non-greedy decision tree / random forest implementation(s) in Python

The standard random forest is trained using a greedy approach for computational feasibility. However, there are a number of alternative methods such as "lookahead" or using bilevel ...
2
votes
0answers
22 views

Classifying short strings of text with additional context

I have a list of short strings each identifying a city. Misspellings are very common. The example below shows some of these short strings, along with the correct city they're supposed to match. ...
0
votes
0answers
7 views

Random Forests notebook collections to be used as benchmark

I'm trying to evaluate my modification of Decision Tree and Random Forests methods as compared to the standard distribution in libraries such as sklearn. I've searched the Kaggle, but did not find a ...
1
vote
1answer
50 views

How do I know if this model is overfitting?

This is my example R script for a decision tree: ...
6
votes
3answers
447 views

Is it possible to build ensemble models without a decision tree?

Is it possible to build ensemble models without a decision tree? I know that the description of ensembles itself suggests otherwise. However, I am really new to machine learning and all the ensemble ...
0
votes
1answer
35 views

Best Way to find the important features for the model [duplicate]

I have data with 245 Features and almost all of the features are categorical. I would like to know what will be the best approach to find the important features for training the model. I know I can ...
1
vote
1answer
77 views

What is the meaning of the Gini Index?

I'm studying random forest models, but I don't understand what the Gini index is and what it's for. Does anyone have any material on this or can give me an explanation? Thanks!
0
votes
1answer
11 views

Additional business rules in ensemble methods (RF, Boosted Trees)

How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees? These additional business rules can be ...
1
vote
0answers
20 views

How are regression trees fitted in gradient boosting for classification?

What I understood is that even gradient boosting for binary classification uses regression trees. The first value we calculate is constant = log(odds). For the rest of the trees, we try to fit ...
1
vote
1answer
43 views

Scikit-learn's implementation of AdaBoost

I am trying to implement the AdaBoost algorithm in pure Python (or using NumPy if necessary)....
0
votes
1answer
27 views

Does hyperparameter tuning of Decision Tree then use it in Adaboost individually vs Simultaneously yield the same results?

So, my predicament here is as follows, I performed hyperparameter tuning on a standalone Decision Tree classifier, and I got the best results, now comes the turn of Standalone Adaboost, but here is ...
2
votes
2answers
34 views

confused on "real score" vs "decision value" in classification trees

I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is: A ...
1
vote
0answers
16 views

Classification for Ordinal labels - what tree-based methds can i use?

I have a label that has a natural ordering e.g. 0,1,2,3 where 0 is the worst activity measure and 3 is the best. For each label given by the model i need to also give the probability that it belongs ...
3
votes
2answers
63 views

Is it possible to 'group features' for a decision tree model?

At each node of a decision tree, we must choose a collection of features to split along. Suppose we know a priori that the features can be partitioned into subsets that are 'correlated', i.e. this ...
0
votes
0answers
20 views

Is it always that lower tree with higher bias but higher tree with higher varaince

When dealing with bias and variance trade-offs, I always hear that in tree models: shallow tree = high bias but low variance, deep tree = low bias but high variance. Someone may also quote from high ...
5
votes
1answer
66 views

Multi-target regression tree with additional constraint

I have a regression problem where I need to predict three dependent variables ($y$) based on a set of independent variables ($x$): $$ (y_1,y_2,y_3) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \...
0
votes
0answers
16 views

User Churn Rate analysis - Binary classification

I have a dataset which has the logs of user clicks. This is a trail version(2 months) of the software. Users can use a special feature during this trail period to improve their sales. The number of ...
0
votes
0answers
14 views

Is there any function of targets in binary classification decision tree

I was recently learning about decision tree and stumbled across a question which might be very silly but i am unable to understand it . That is if for a binary classification problem splits are used ...
0
votes
1answer
63 views

Orthogonal Decision Boundary in Decision Tree

I was reading the limitations of decision trees. One of them was that, for classification problems, decision trees produce only orthogonal decision boundaries. Could anyone please explain what an ...
2
votes
0answers
22 views

Building machine learning models whilst penalizing them for complexity

I come from a predictive modelling background, where it's common to use differential equations to model physical or chemical or biological processes. Commonly to avoid overfitting people use AIC and ...
0
votes
2answers
29 views

How to create classification decision trees on a dataset that has both numerical and categorical variables?

I am quite new to Data Science and learning things hands-on in the job. I am a fraud analyst and my job is to predict whether an application is fraudulent or not based on data. Before moving on to ...
0
votes
0answers
29 views

LightGBM boosting and bagging parameters

When training a gradient boosted decision tree model, I can use the LightGBM package to efficiently train my model. It's possible to define the hyperparameter search space with eg. ...

1
2 3 4 5
14