Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
0 votes
1 answer
10 views

Calculating feature importance with Scikit-Learn's Decision Tree Classifier

I am attempting to determine the most useful bands of a multiband image classification (i.e. Red, Green, Blue, Near Infrared, etc. used for classifying pixels) and wrote the following function to ...
camdenmcgath's user avatar
0 votes
1 answer
15 views

How to represent varying reliability of ratios calculations in a dataset?

I want to predict whether the client will renew his/her subscription based on groceries consumption patterns. Suppose an order contain only one type of grocery. I have a DataFrame containing ratios of ...
a_long_road_ahead's user avatar
1 vote
1 answer
26 views

Distilling a Random Forest to a single DecisionTree, does it make sense?

I stumbled into this blog which shows how a decision tree trained to overfit the predictions of a properly trained random forest model is able to generalize in pretty much the same way as the original ...
amiando's user avatar
  • 35
0 votes
1 answer
14 views

How do I use a column with data of different layers for AI?

I am working with real estate data for an ML/DL project. In the csv file there is a column in which each cell contains data like the examples below: ...
Muhammad Usman's user avatar
0 votes
0 answers
9 views

Feature scales and feature importance

Tree-based algorithms do not require feature scaling before fitting, and I am working on gradient boosted tree models (and random forest) without scaling features. I'm curious if feature scaling ...
Matthew Son's user avatar
0 votes
0 answers
8 views

Overlapping values of a variable in decision tree

Is it okay to have a variable, this variable has values which are subsets of others in the same variable when building a decision tree? To be specific, I am working with a dataset that have a variable ...
MINH NHỰT NGUYỄN TRẦN's user avatar
0 votes
0 answers
5 views

Value[] attribute in my decision tree is not consistent with number of samples

I read that value[] attribute in a decision tree shows the distribution of the samples across class 1 and class 2. However, my value[] is not adding up. In the root node for example, there are 14 ...
Dharmini's user avatar
1 vote
2 answers
108 views

Can decision trees handle Nominal Categorical variables?

I have read that decision trees can handle categorical columns without encoding them. However, as decision trees make splits on the data, how does it handle Nominal Categorical variables? Surely a ...
Connor's user avatar
  • 557
0 votes
1 answer
23 views

DART algorithm implementation. Converting mathematical notation to pseudocode

I am learning how DART algorithm (https://arxiv.org/abs/1505.01866) works and I want to implement it in C# I have the algorithm's description in mathematical notation and I don't understand most of it....
michaelfromsomeplace's user avatar
0 votes
1 answer
19 views

XGBoost prints trees beyond n_estimator

I have a XGBoost model with the following parameters ...
Itminan's user avatar
0 votes
1 answer
22 views

Noisy Data Robustness - NN vs Decision Tree

We are working on a physiological marker predictor using hospital patient data. We use a boosted decision tree-type algorithm, which seems to be very sensitive to the noise in the training data. Would ...
machinelearner's user avatar
0 votes
0 answers
15 views

Is it possible to capture time-specific outliers using a Decision Tree?

Having this data set ...
Carlos Navarro Astiasarán's user avatar
0 votes
1 answer
50 views

how do I test if overfitting exists when I use cross_val_score method?

I got the following code form a book on xgboost. I wonder whether this is a correct way of analyzing cross validation score for overfitting purposes. mean accuracy is 81 which can be okay. but what if ...
Mehmet Deniz's user avatar
0 votes
1 answer
143 views

How do the splits points in a decision tree within Random Forest are taken/selected? (Base on which criteria?)

I checked many posts to figure out how random forest (RF) learning algorithm (an ensemble of many decision trees (DT) constructed by Rain forest algorithm) within bagging select split points at each ...
Mario's user avatar
  • 335
2 votes
1 answer
52 views

Surrogate splits in Python

I want to use RandomForestClassifier from Sklearn to predict categorical variable (credit risk). But one of the predictors seems to have missing values: ...
Ars ML's user avatar
  • 61
2 votes
1 answer
64 views

help interpreting training/validation curves for classification tree

I'm developing a binary classification tree and having some touble interpreting my training/validation curves. I used the CART algorithm with information gain as my splitting criterion. The training ...
RyRy the Fly Guy's user avatar
0 votes
0 answers
13 views

Why are decision trees driven by the Gini impurity as opposed to the accuracy? [duplicate]

It seems that most implementations of decision trees use the Gini impurity as their partitioning criterion. Why isn't accuracy used instead, since it's a more widespread metric across different ...
Tfovid's user avatar
  • 203
0 votes
1 answer
41 views

DecisionTreeClassifier cannot take one-hot encoded classes?

I got ValueError: Found array with dim 3. None expected <= 2. I dont know which array has dim 3? DecisionTreeClassifier cannot take one-hot encoded classes? But ...
user900476's user avatar
0 votes
0 answers
25 views

About feature importance in deep learning

For tree methods, I can plot the feature importance plot from tree.feature_importances_ in sklearn, is this achievable in deep learning (neural networks)? Is there ...
user900476's user avatar
1 vote
0 answers
28 views

DecisionTreeRegressor with criterion='poisson' not recognizing perfect separation

I created a minimal example of Poisson decision tree regression as such ...
Jason Hadinata's user avatar
1 vote
1 answer
27 views

How to use labels to fit several thresholds in a simple decision rule?

I have a binary labelled dataset with numeric features. I want to create a "business rule" of the type y = x1 > t1 and x2 > t2 and x3 > t3. ...
hipoglucido's user avatar
  • 1,160
0 votes
0 answers
9 views

How to compensate for different feature sampling in decision trees

I have a dataset, on which I would like to use a decision tree, where some features are sampled much less frequently than others. I am concerned that they could lead to suboptimal feature selection in ...
Andy's user avatar
  • 1
0 votes
0 answers
21 views

Can the product of tree regressions be represented by a single tree?

Assume that we have two separate tree regressions. I'm interested in understanding whether the product of tree regressions can be represented by a single tree. Would this be possible?
TFT's user avatar
  • 23
0 votes
0 answers
13 views

What is the use of validation dataset when doing regression-based outlier detection?

I have a dataset where data are velocity data splitted as: 60%(train - non-anomalous) 20%(validation - 50% of it anomalous) 20%(test - 50% of it anomalous). From my understanding, when doing outlier ...
Mr. Panda's user avatar
  • 121
4 votes
2 answers
242 views

Loss for ordered multi class data in classification

Assume data which is labeled $y_i \in \left\{ 1, 2, 3, \ldots, 9, 10\right\}$. Assume the labels are ordered, namely, given $y_i = 10$ to estimate $\hat{y}_{i} = 1$ is much worse than $\hat{y}_{i} = ...
GeorgeIrwin's user avatar
0 votes
0 answers
56 views

Best way to handle missing values with XGBoost?

I know of a number of ways to handle missing feature values, and wanted to get folks' input on what might work best. My end goal is to be able to predict accurate probabilities for a binary ...
hologram's user avatar
  • 101
0 votes
0 answers
10 views

Build decision tree with fixed features, but learn optimal value of features?

I'm trying to build a hybrid between an expert system and a decision tree model to serve as a baseline for transparent comparison with more sophisticated models. My problem is as follows: I have a ...
JuRoSch's user avatar
0 votes
1 answer
100 views

Decision tree vs logistic regression feature importances

I have trained Logistic regression and decision tree in skearn on the same standardized dataset (binary classification). Top important coefficients for the decision tree are (sorted by ...
Arseniy Maryin's user avatar
0 votes
0 answers
26 views

Understanding Cost-sensitive Decision Trees Behaviour

I have a binary classification problem with imbalanced data and am attempting to use cost-sensitive learning to handle the imbalance. I have used LogisticRegression, LinearSVC, SVC and DecisionTree ...
sums22's user avatar
  • 387
3 votes
1 answer
203 views

Why am I getting the exact same results with both a Logistic Regression and Decision Tree Classifier?

I am working on a binary classification problem and am using sklearn's logistic regression model and decision tree classifier. Somehow I am getting the exact same results and accuracy score on both. I ...
RandomGuy's user avatar
0 votes
0 answers
25 views

My overfitted decision tree regressor gives better result than pre-pruned tree?

I create a decision tree regressor without giving any parameters. The resulting tree has 6255 leaf nodes (out of 6348 entries of train set) and depth of 39. Most probably it has overfitted. But its ...
Akrobeto's user avatar
1 vote
1 answer
46 views

N-ary decision tree with categorical features

I want to build an n-ary decision tree with categorical features. I am using ordinary ID3 algorithm to build a tree. Lets take the next dataset as a training dataset for building a decision tree: ...
dzi's user avatar
  • 111
0 votes
1 answer
200 views

How can I train a decision tree constrained to have number of decision nodes = tree depth?

In order to make a classifier dead easy to understand/interpret, I want to classify tabular data (with n columns) according to a set of nested rules, with the ...
Davide Fiocco's user avatar
0 votes
1 answer
23 views

How do I use one hot encoding with 240 nominal variables and each with equal occurrence?

The method I saw that's generally used to deal with large # of nominal variables is to keep the most frequent variables and introduce a new "other" category. But that's not possible with my ...
learner's user avatar
0 votes
1 answer
27 views

How to properly use regression / tree based models for time-series data

Regression/tree-based models appear to treat each prediction as a memoryless process, namely given a feature vector $\hat{x}_i$, predict $y_{i+1}$, but previous states $\hat{x}_{i-1}$, $\hat{x}_{i-2}, ...
ron burgundy's user avatar
0 votes
0 answers
30 views

When you plot features(pairplot) and gets many vertical plots, which non-linear regression would be useful to make sense of it?

I have plotted my N features against the output and I have multiple vertical plots and was wondering, if I should go by decision trees.
Rookie91's user avatar
  • 101
0 votes
0 answers
17 views

compare decision tree vs extended gradient boosting mathematically?

If we want to compare decision tree vs extended gradient boosting vs xgboost mathematically, what are their differences?
user10296606's user avatar
  • 1,754
0 votes
0 answers
36 views

What is the difference between using Minimal Cost-Complexity Pruning and testing all possible tree depths in a decision tree?

I´m studying the sklearn decision tree classifier and I´m having some trouble understanding the concept of pruning. From what I understand it consists in making the tree less deep in order to avoid ...
Ajoa's user avatar
  • 1
0 votes
2 answers
47 views

Can Gradient Descent boosted decision trees miss the forest for the trees?

My understanding of this stuff is pretty basic so my semantics may be off, but bare with me. XGBoost and other gradient descent packages make the best possible split of the data right off the bat. ...
helloimgeorgia's user avatar
0 votes
1 answer
322 views

Splitter in decision trees in sklearn implementation

I am very confused about how decision trees select features and threshold within each feature to do the split. I totally understand the different splitting metrics (Gini index and so on) used and how ...
AAA's user avatar
  • 35
0 votes
0 answers
57 views

is time series data normalization for xgboost required?

According to the developer - xgboost does not require feature normalization https://github.com/dmlc/xgboost/issues/357 no you do not have to normalize the ...
Alex's user avatar
  • 101
0 votes
1 answer
98 views

how to compute the possible number of splits in decision tree?

In the following dataset, if we want to include just two variables, STORE and PctDiscMM, in a classification tree model, what is the possible number of first splits? ...
ebrahimi's user avatar
  • 1,277
1 vote
4 answers
309 views

Draw a decision tree with depth 2 that is consistent with the data

I am trying to come up with a solution to this for an exam preparation but cant come up with anything, dont know how to tackle it... if i use information gain the depth increases beyond 2. What would ...
fearloathing121's user avatar
2 votes
1 answer
59 views

Model from an aggregate

I’m in a place where we’re unable to train models on data due to GDPR. What I want is to predict people getting a job (y) given (x,x,x,x…) their employment type working full time or part time, work ...
SamTheGoodOne's user avatar
1 vote
1 answer
321 views

Online Learning/Continual Learning for tree-based Algorithms

Every example I come across any kind of iterative learning on Random Forest/XGBoost/LightGBM, it just continuously grows the number of estimators for new batches of data by ...
OliverHennhoefer's user avatar
0 votes
1 answer
975 views

I am facing the error of DecisionTreeregressor object has no attribute n_features

...
gaurav's user avatar
  • 1
1 vote
0 answers
8 views

Precision and AUROC for which class values

I am a newbie in reading research paper and implementing it by myself. I went through the paper Breast Cancer Survival Prediction from Imbalanced Dataset with Machine Learning Algorithms. Can anyone ...
Encipher's user avatar
  • 359
1 vote
1 answer
191 views

Decision tree implementation in python that correctly handles categorical variables

Is there a Decision tree implementation in python that correctly handles categorical covariates? By "correct" I mean that it is able to send any subset of category levels down one daughter ...
Iyar Lin's user avatar
  • 749
1 vote
0 answers
26 views

ML Model that doesnt average/penalize extreme values

I have a 20k dataset, and a couple hundred of those lines are extreme values and 10 of them or so are even extremer values. But they are correct and have a unique tag, so when that tag comes up I am ...
Jroc561's user avatar
  • 11
0 votes
0 answers
31 views

Is it possible to extract precise decision boundaries from a random forest for a multiclass classification?

I have a random forest (argmax post-processor) with 3 trees and 10 input features. The final outcome of the random forest is either true or false depending on the combination of the feature values. Is ...
giantjenga's user avatar

1
2 3 4 5
15