2022 Developer Survey is open! Take survey.

Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

Do cost-complexity pruned trees perform better relative to unpruned trees?

I came across an adapted question from the famous ISLR book and realise I am unsure of the answer. Does anyone know? Interested in the intuition here! Cost-complexity pruned trees with $\alpha=1$ ...
user avatar
1 vote
2 answers
23 views

How do I design a random forest split with a "not sure" category?

Let's say I have data with two target labels, A and B. I want to design a random forest that has three outputs: A, B and Not sure. Items in the Not sure category would be a mix of A and B that would ...
user avatar
  • 249
0 votes
1 answer
19 views

How does ExtraTrees (Extremely Randomized Trees) learn?

I'm trying to understand the difference between random forests and extremely randomized trees (https://orbi.uliege.be/bitstream/2268/9357/1/geurts-mlj-advance.pdf) I understand that extratrees uses ...
user avatar
  • 249
1 vote
0 answers
11 views

Machine learning frameworks for tree-based models

Background: Its well known that Pytorch and TensorFlow are currently the most used frameworks for Deep Learning (DL) research. As far as I know, most researchers (applied or theoretical) that ...
user avatar
  • 89
0 votes
1 answer
10 views

forcing decision tree use specific features first

My goal it to force some feature used firstly to split tree. Below, the function splitted tree using feature_3 first. For instance, is there a way to force to use feature_2 first instead of feature_3 ?...
user avatar
  • 105
0 votes
0 answers
15 views

Information gain calculation for decision tree

I am having trouble figuring out how to calculate information gain in decision trees. Suppose a decision tree node splits the data of 5 red and 5 black balls into 2 branches: 4 reds in the right child ...
user avatar
  • 1
0 votes
0 answers
46 views

What is the max number of leaf nodes in a classifcation decision tree?

Let's assume that we have n observations and p predictors and we have in a n>>p situation. All predictors are binary. What is the max number of leaf nodes that we can have in the tree? and what ...
user avatar
0 votes
0 answers
20 views

Decision tree from boosted tree regressor Google bigquery ML

If I set the num_parallel tree to 1 and max_iteration to 1 in boosted_tree_regressor of Google Big Query ML will it work as Decision tree regressor ? Also can such decision tree give negative ...
user avatar
0 votes
1 answer
26 views

Identifying subsets of values significant to the total sum

Imagine a set of products in a store, with all the different attributes assigned to them - some of these hierarchical (e.g. categories), and some not (e.g. brand), but none of them continuous (if that ...
user avatar
1 vote
1 answer
60 views

How to deal with missing values that are supposed to be missing?

I am trying to predict loan defaults with a fairly moderate-sized dataset. I will probably be using logistic regression and random forest. I have around 35 variables and one of them classifies the ...
user avatar
  • 13
0 votes
1 answer
37 views

How to compute the Gini index, the entropy and the classification error from a decision tree?

How to find the Gini index, the entropy, and the classification error for each node of the tree in the figure below. Please help me to compute them.
user avatar
  • 21
0 votes
0 answers
4 views

Neural Networks that can mimic Decision Trees

tl;dr: Are there neural networks (specific activation functions, set up of layers ...?) that can mimic fairly well decision trees/if statements? === I am trying to build a model from some simulated ...
user avatar
  • 21
0 votes
0 answers
13 views

Resultant entropy of the target feature example

I'm confused by an example I have come across on entropy. In a decision tree, we have after a split on some particular feature, the following subset of our training data. What is the resultant ...
user avatar
0 votes
1 answer
25 views

Why my models have a pretty high accuracy with a small training dataset?

I was wondering why my models (decision tree, svm, random forest) behave like that, with "high" accuracy on a small training dataset. Is it a sign of overfitting? The graph represents the ...
user avatar
0 votes
1 answer
35 views

Decision Tree: SPRINT vs SLIQ?

I found different types of decision tree, for example SPRINT and SLIQ methods. Both methods are used for classification problem, use Gini Index for the feature selection and follow the procedure (...
user avatar
  • 411
0 votes
0 answers
26 views

How to make TensorFlow Decision Forests models compatible with scikit-learn?

I am trying to create an ensemble using a tensorflow_decision_forests.keras.CartModel (from TensorFlow Decision Forests) as the ...
user avatar
1 vote
1 answer
18 views

Not perfect accuracy when overfitting

Given a dataset and a decision tree that can be as depth as it wants, if you train the tree with all the dataset and then you test it against the same dataset and you get an accuracy that is not 100%, ...
user avatar
0 votes
2 answers
43 views

What kind of decision tree are used in random forest?

Read some documentation (for example) I know that there are many types of decision tree (Cart, ID3 and so on). I also know that Random Forest is a particolar algorithm that use a set of decision tree. ...
user avatar
  • 411
1 vote
0 answers
15 views

Regression tree cross validation confusing results

Setup I have implemented regression trees in go: full repository Using the full dataset and cost-complexity pruning, I get the following alphas and corresponding average residual (again, against the ...
user avatar
0 votes
0 answers
12 views

Is there a random forest env (sci-kit, TFDF, R, etc) that has an implementation for multi-output regression?

It is easy to adapt the idea of tree based linear regression to perform logistic regression: The decision boundaries of the tree divide the space of independent variables into hyper-cubes, and each ...
user avatar
0 votes
0 answers
15 views

Policy from an ensemble tree model?

I'm trying to output a policy (pdf on a fixed number of finite labels) from a Random Forest model. What is the best way to achieve this? I'm thinking that I can just normalize the votes from all the ...
user avatar
2 votes
2 answers
200 views

How Decision Tree Classifier works? [closed]

In particular i am using SKLearn with class DecisionTreeClassifier. I would really like to understand how the tree build itself ...
user avatar
0 votes
0 answers
16 views

Struggling with decision tree classification SKLearn and MultiLabelBinarizer [duplicate]

So i have training data like this: ...
user avatar
0 votes
0 answers
7 views

Extending a classifier with specialised features

Let's say we have an app and a classifier (GBDT) predicting whether a user is a good user or bad (whatever that means) based on generic signals that every user has like profile fields, how long they ...
user avatar
0 votes
1 answer
46 views

Low accuracy on the test set

I have a dataset with 16 features and 32 class labels, which shows the following behavior: Neural network classification: high accuracy on train 100%, but low accuracy on the test set 3% (almost like ...
user avatar
  • 163
0 votes
1 answer
31 views

Does it make sense to add a new calculated column for dates/duration?

I'm using a Random Forest Classifier on some data, and I have two date field, StartDate and EndDate. Does it make sense to ...
user avatar
  • 1,248
0 votes
0 answers
10 views

Best way to manually traverse hierarchical decision trees in R

I am currently building an application that allows users interact with tree models to generate classification rules that fit their specified parameters. The idea is simple: given any set of rules ...
user avatar
  • 1,621
0 votes
0 answers
21 views

Simple way to parse the rules of a Tree model

Problem Description I am currently working with simple decision tree models to generate rules to classify respondents of a survey. The output of each model is "translated" to the complete ...
user avatar
  • 1,621
0 votes
1 answer
28 views

Give more weight to features based on distribution plot

I have a task to predict a binary variable purchase, their dataset is strongly imbalanced (10:100) and the models I have tried so far (mostly ensemble) fail. In ...
user avatar
  • 101
0 votes
1 answer
32 views

Simple CART model example

My goal is to test Decision tree to regression model. My data is like below(python dataframe). There are 2 features F1 and F2. And there is label which is number. How to make CART model from this ...
user avatar
  • 105
0 votes
0 answers
42 views

Getting entropy in decision trees more than 1

My decision tree entropy is coming more than 1 when I'm calculating it manually. Not sure if there's some calculation error. Trying it on the Iris dataset. If I split on sepal length at 6.5 cm, my ...
user avatar
0 votes
1 answer
38 views

Why -2 is seen in supervised binning using decision tree?

I have a continuous variable called salary, age etc and output variable as loan_status Instead of me choosing the cut off points for salary and age bins , I used Decision Tree to compute the bins ...
user avatar
  • 2,129
4 votes
1 answer
478 views

Are linear models better when dealing with too many features? If so, why?

I had to build a classification model in order to predict which what would be the user rating by using his/her review. (I was dealing with this dataset: Trip Advisor Hotel Reviews) After some ...
user avatar
  • 151
1 vote
1 answer
33 views

Understanding feature_parallel distributed learning algorithm in LightGBMClassifier

I want to understand feature_parallel algorithm in LightGBMClassifier. It describes how it is done traditionally and how LightGBM...
user avatar
0 votes
1 answer
81 views

SKLearn decisionTreeClassifier does not handle sparse or categorical data

Is there a way in fitting a decisionTreeClassifier in SKLearn to sparse tuples? The data that I have is based on about 100 features, but only a few of them are ever used to make the decision. ...
user avatar
  • 186
0 votes
1 answer
79 views

Interpreting 'values' of a Decision Tree

I am trying to interpret my decision tree here which was resulted as a part of pre-pruning- I am trying to understand why the values in my nodes are in decimal places. Ideally, they should represent ...
user avatar
  • 23
0 votes
0 answers
19 views

Approximating linear and non-linear functions using 1-depth trees

I'm taking a course in statistical learning and we learned about tree models, namely decision trees. Now, I'm preparing for the test and there are two questions from past exams: Given $y=x_1+2x_2$ ...
user avatar
  • 1
0 votes
0 answers
26 views

Where can I find the original CART(classification and regression trees) published paper?

I was trying to find the original CART paper. I found papers like https://www.researchgate.net/publication/227658748_Classification_and_Regression_Trees which experimented on CART but was unable to ...
user avatar
0 votes
0 answers
12 views

Optimize parametric Log-Likelihood with a Decision Tree

There are some objects with features, and the target is parametric estimation for some model-based variables. Model parameters $\theta_j$ are obtained by maximizing log-likelihood. $LL = \sum_{i \in ...
user avatar
0 votes
0 answers
19 views

R: Handling tags in R rpart

I'm new to regression trees and wanted some advice. I'm working on a data set in which a few of the columns contains tag-like information (basically tags separated by commas). My goal is to create a ...
user avatar
  • 1
3 votes
2 answers
77 views

How to control a decision tree?

This is my R script for a decision tree: ...
user avatar
  • 411
0 votes
1 answer
26 views

How to drawing a decision tree?

This is my script for a decision tree in R: ...
user avatar
  • 411
0 votes
1 answer
68 views

Tree based Classifiers with Label Encoder and One Hot Encoder [duplicate]

I m working with Tree-based classifiers in scikit-learn - Decision Trees and Random Forest, for a data classification use case, and the feature set is a mix of both categorical (majority) and ...
user avatar
  • 101
0 votes
1 answer
21 views

Error with decision tree prediction

I write this script in R about decision tree. ...
user avatar
  • 411
1 vote
1 answer
27 views

Reasons for a model predicting probability of being class 1 at x value

All, This is a general question. I have a binary classification which predicts if someone is rich or not. I had a question from someone asking that if the probability someone is rich is 0.6 and ...
user avatar
  • 466
0 votes
0 answers
15 views

Looking for an algorithm to perform classification on multivariate grouped time series

I will be grateful for any help. I have multivariate time series, where every one of them has an unique ID. Also, there is a variable giving information about the trend type of the ID from a point of ...
user avatar
0 votes
0 answers
20 views

Handling repeating data from different individuals

I have a dataset that has some unique values but also includes information from multiple individuals that are repeating, meaning they are describing the same attributes and can have the same or ...
user avatar
0 votes
0 answers
21 views

How to apply post pruning methods to ID3 decision tree

I am developing an ID3 decision tree implementation that feature post-pruning and classification. The program below constructs the decision tree. ...
user avatar
0 votes
0 answers
11 views

Decision Trees and SHAP Values

I've recently been using some (optimal) decision trees methods in R, such as 'evtree' and 'iai.' Both of these provide really nice interpretable plots. And out of the 12 covariates I have in my model, ...
user avatar
  • 193
0 votes
0 answers
13 views

How to extract rules for decision tree from ID3 classification

I am working on a program to implement the ID3 algorithm. The program takes in user input for setting a threshold and creating a decision tree. ...
user avatar

1
2 3 4 5
14