Questions tagged [decision-trees]
A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.
749
questions
0
votes
0
answers
9
views
Finding Accuracy, Recall, Precision, and F1 from Matlab Confusion Matrix
I'm working on a project to find the highest accuracy between KNN and a Decision Tree for Classification using Matlab.
How to calculate the Accuracy, Recall, Precision, and F1 from the output below? ...
- 1
0
votes
0
answers
5
views
Oblique Decision Tree HHCART algorithm explaination
I am now studying a paper about implementing oblique decision tree algorithm
https://arxiv.org/abs/1504.03415
I was wondering if any code implementation and alternative explaination, as a support ...
- 149
4
votes
1
answer
720
views
Decision Tree only splits to the left
I can’t really understand, why my decision tree only splits to the left.
I originally have 2 categorical features (further named feature 0 and 1), which I concat to one feature since feature 1 is ...
- 41
0
votes
0
answers
11
views
Do outlier data in target variable cause a problem for GBM's?
I wonder if I should exclude outlier (legit data, not wrong readings) data from my dataset using gradient boosting.
Let's say we try to predict water damage for regular houses and 99% of data is in 0-...
- 25
0
votes
0
answers
14
views
decision tree limitation VS deep learning
I wonder if decision trees (and their derivatives like Random Forest and Gradient Boosting) have interpolation power as deep learning based model. Most of my experience is with deep learning model.
...
- 101
0
votes
1
answer
60
views
ValueError: X has 54 features, but DecisionTreeClassifier is expecting 53 features as input
I am analysing and prediction 2023 Cricket World Cup based on previous given dataset.
This is Exploratary analysis:
Feature selection and Training model:
Applying Random forest classifier algorithm:
...
0
votes
1
answer
56
views
Optimizing decision tree
I have a question regarding the technique/technology which could be applied for the issue:
Suppose I have a rule-based tree or decision tree which predicts a variable Y based on variables A,B,C. This ...
- 1
1
vote
1
answer
190
views
How to calculate the training accuracy of a decision tree?
The hint given was to construct a confusion matrix.
1
vote
0
answers
23
views
How to know which rules were applied to predict one sample in trained decision tree model?
I have trained Random Forest Regressor from sklearn. I am able to return text representation from each Decision Tree rule using tree.export_text (sklearn documentation here). But it shows rules for ...
- 15
0
votes
0
answers
8
views
decision tree and prediction
A decision tree regression model produces at most as many possible predictions as it has terminal nodes/leaves. A well fit decision tree will have been limited in the number of terminal nodes/leaves, ...
0
votes
0
answers
13
views
p value and decision tree
let's say you create a GLM with y~x1+x2+x3. The p-value for x1 is 1% and x2 is 5% and x3 is 10%. Then, you know that beta for x1 and x2 are reliable, but not with x3. So, you can disregard x3's ...
0
votes
0
answers
27
views
Training Biased/Uneven Categorical Data with CatBoost, Unbalanced/Unseen Categories Handling
Summary:
I am training a discount eligibility model where the dataset represents historical data for products where people availed discounts based on simple features like product category, discount ...
- 101
0
votes
0
answers
42
views
One-Hot encoded variables dominates importance among other variables
I am currently training some machine learning models to predict the 28-day compressive strength of cement, a continuous real-valued variable. The available dataset comprises samples from three ...
- 11
-1
votes
2
answers
71
views
Recall and Precision ML models
I use decision trees for a binary classification. To evaluate the model, I use K-fold cross-validation, where k = 10.
When I run the model n times, I get a relatively constant accuracy across all ...
1
vote
1
answer
24
views
Regression models that conform to functional groupings of features
For example, suppose we want to predict y with features x1, x2, x3, x4. If I specify
...
- 11
0
votes
0
answers
41
views
How to predict an xgboost model outcome directly from the trained trees?
I want to train by Xgboost algorithm and predict directly using the trees while testing. Precisely, speaking I don't want to keep the model weights in any file like "joblib" and load it ...
- 103
0
votes
1
answer
83
views
How does LGBM make a prediction?
We are currently trying to figure out how LGBM creates its trees and how predictions are made afterwards.
In my current understanding, it works as follows:
Multiple "weak learners" are ...
- 121
0
votes
1
answer
62
views
Does Random Forest Regressor use subset of trees to predict value from given data sample?
I will try to draw a little context to my question from title.
I build a Random Forest Regressor from 1000 trees using sklearn. Then I exported all the decision paths along with predicted values for ...
- 15
0
votes
2
answers
45
views
how to deal if input feature are all categorical but target feature is discrete numerical, it is giving exact numerical for known data
the following data is to detect malnutrition among children under age 5 and the value_r is the percentage estimate of wasting among the population. should I apply the decision tree as an entirely ...
- 1
0
votes
1
answer
12
views
Decision tree classification code for categorical data
how can I apply decision tree classification to get malnutrition status(target variables are wasting, stunting,overweight,underweight)
- 1
0
votes
1
answer
67
views
Decision Tree : why do unused features impact predictions?
I am currently building a tree, with 10 features but setting max_depth = 2 in sklearn.tree.DecisionTreeClassifier.
Since only ...
- 101
2
votes
1
answer
133
views
Difference among ID3, C4.5, C5.0
The C4.5 algorithm uses information gain ratio instead of information gain like ID3, and it also adds pruning.
What does C5.0 add more? Is there any example of code? I looked on the web but there is ...
- 152
0
votes
1
answer
147
views
Calculating feature importance with Scikit-Learn's Decision Tree Classifier
I am attempting to determine the most useful bands of a multiband image classification (i.e. Red, Green, Blue, Near Infrared, etc. used for classifying pixels) and wrote the following function to ...
0
votes
1
answer
17
views
How to represent varying reliability of ratios calculations in a dataset?
I want to predict whether the client will renew his/her subscription based on groceries consumption patterns. Suppose an order contain only one type of grocery.
I have a DataFrame containing ratios of ...
1
vote
1
answer
56
views
Distilling a Random Forest to a single DecisionTree, does it make sense?
I stumbled into this blog which shows how a decision tree trained to overfit the predictions of a properly trained random forest model is able to generalize in pretty much the same way as the original ...
- 35
0
votes
1
answer
19
views
How do I use a column with data of different layers for AI?
I am working with real estate data for an ML/DL project. In the csv file there is a column in which each cell contains data like the examples below:
...
0
votes
0
answers
20
views
Feature scales and feature importance
Tree-based algorithms do not require feature scaling before fitting, and
I am working on gradient boosted tree models (and random forest) without scaling features.
I'm curious if feature scaling ...
- 101
0
votes
0
answers
30
views
Overlapping values of a variable in decision tree
Is it okay to have a variable, this variable has values which are subsets of others in the same variable when building a decision tree?
To be specific, I am working with a dataset that have a variable ...
0
votes
0
answers
18
views
Value[] attribute in my decision tree is not consistent with number of samples
I read that value[] attribute in a decision tree shows the distribution of the samples across class 1 and class 2.
However, my value[] is not adding up. In the root node for example, there are 14 ...
- 1
1
vote
2
answers
601
views
Can decision trees handle Nominal Categorical variables?
I have read that decision trees can handle categorical columns without encoding them.
However, as decision trees make splits on the data, how does it handle Nominal Categorical variables?
Surely a ...
- 617
0
votes
1
answer
58
views
DART algorithm implementation. Converting mathematical notation to pseudocode
I am learning how DART algorithm (https://arxiv.org/abs/1505.01866) works and I want to implement it in C#
I have the algorithm's description in mathematical notation and I don't understand most of it....
1
vote
1
answer
25
views
0
votes
1
answer
81
views
Noisy Data Robustness - NN vs Decision Tree
We are working on a physiological marker predictor using hospital patient data. We use a boosted decision tree-type algorithm, which seems to be very sensitive to the noise in the training data. Would ...
0
votes
0
answers
16
views
Is it possible to capture time-specific outliers using a Decision Tree?
Having this data set
...
0
votes
1
answer
59
views
how do I test if overfitting exists when I use cross_val_score method?
I got the following code form a book on xgboost.
I wonder whether this is a correct way of analyzing cross validation score for overfitting purposes.
mean accuracy is 81 which can be okay. but what if ...
- 31
0
votes
1
answer
568
views
How do the splits points in a decision tree within Random Forest are taken/selected? (Base on which criteria?)
I checked many posts to figure out how random forest (RF) learning algorithm (an ensemble of many decision trees (DT) constructed by Rain forest algorithm) within bagging select split points at each ...
- 364
2
votes
1
answer
193
views
Surrogate splits in Python
I want to use RandomForestClassifier from Sklearn to predict categorical variable (credit risk). But one of the predictors seems to have missing values:
...
- 61
2
votes
1
answer
123
views
help interpreting training/validation curves for classification tree
I'm developing a binary classification tree and having some touble interpreting my training/validation curves. I used the CART algorithm with information gain as my splitting criterion. The training ...
- 121
0
votes
0
answers
14
views
Why are decision trees driven by the Gini impurity as opposed to the accuracy? [duplicate]
It seems that most implementations of decision trees use the Gini impurity as their partitioning criterion. Why isn't accuracy used instead, since it's a more widespread metric across different ...
- 203
0
votes
1
answer
72
views
DecisionTreeClassifier cannot take one-hot encoded classes?
I got ValueError: Found array with dim 3. None expected <= 2. I dont know which array has dim 3?
DecisionTreeClassifier cannot take one-hot encoded classes?
But ...
- 165
0
votes
0
answers
72
views
About feature importance in deep learning
For tree methods, I can plot the feature importance plot from tree.feature_importances_ in sklearn, is this achievable in deep learning (neural networks)?
Is there ...
- 165
2
votes
1
answer
81
views
DecisionTreeRegressor with criterion='poisson' not recognizing perfect separation
I created a minimal example of Poisson decision tree regression as such
...
1
vote
1
answer
28
views
How to use labels to fit several thresholds in a simple decision rule?
I have a binary labelled dataset with numeric features. I want to create a "business rule" of the type y = x1 > t1 and x2 > t2 and x3 > t3. ...
- 1,170
0
votes
0
answers
9
views
How to compensate for different feature sampling in decision trees
I have a dataset, on which I would like to use a decision tree, where some features are sampled much less frequently than others. I am concerned that they could lead to suboptimal feature selection in ...
- 1
1
vote
1
answer
36
views
Can the product of tree regressions be represented by a single tree?
Assume that we have two separate tree regressions. I'm interested in understanding whether the product of tree regressions can be represented by a single tree. Would this be possible?
- 35
0
votes
0
answers
14
views
What is the use of validation dataset when doing regression-based outlier detection?
I have a dataset where data are velocity data splitted as:
60%(train - non-anomalous)
20%(validation - 50% of it anomalous)
20%(test - 50% of it anomalous).
From my understanding, when doing outlier ...
- 131
3
votes
2
answers
406
views
Loss for ordered multi class data in classification
Assume data which is labeled $y_i \in \left\{ 1, 2, 3, \ldots, 9, 10\right\}$.
Assume the labels are ordered, namely, given $y_i = 10$ to estimate $\hat{y}_{i} = 1$ is much worse than $\hat{y}_{i} = ...
- 131
0
votes
0
answers
180
views
Best way to handle missing values with XGBoost?
I know of a number of ways to handle missing feature values, and wanted to get folks' input on what might work best. My end goal is to be able to predict accurate probabilities for a binary ...
- 101
0
votes
0
answers
19
views
Build decision tree with fixed features, but learn optimal value of features?
I'm trying to build a hybrid between an expert system and a decision tree model to serve as a baseline for transparent comparison with more sophisticated models. My problem is as follows:
I have a ...
- 1
0
votes
1
answer
663
views
Decision tree vs logistic regression feature importances
I have trained Logistic regression and decision tree in skearn on the same standardized dataset (binary classification).
Top important coefficients for the decision tree are (sorted by ...
