Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

162 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
80 views

How is bayesian risk computed to prune decision trees?

I've been trying to follow this paper on Bayesian Risk Pruning. I'm not very familiar with this type of pruning, but I'm wondering a few things: (1) The paper describes risk-rates to be defined per ...
4
votes
1answer
857 views

Is there a real C4.5 implementation in Python ? (handling missing value)

To my understanding, C4.5 comes with 4 improvements compared to ID3: Handling missing values in both training data and "test" data, Handling continuous data Handling costs on attributes. The pruning ...
4
votes
0answers
3k views

Isolation Forest Feature Importance

As of scikit-learn version 0.19.1, there is no implementation for calculating feature importance in an Isolation Forest. I'm also having trouble finding any online resources proposing ways to get at ...
4
votes
0answers
2k views

Decision Trees - C4.5 vs CART - rule sets

When I read the scikit-learn user manual about Decision Trees, they mentioned that CART (Classification and Regression Trees) is very similar to C4.5, but it differs in that it supports numerical ...
4
votes
0answers
835 views

How does XGBoost compute the probabilities in predict_proba()?

I'm using the sklearn wrapper for XGBoost. I didn't manage to find a clear explanation for the way the probabilities given as output by predict_proba() are computed. In random forest for example, I ...
4
votes
2answers
3k views

fix first two levels of decision tree?

I am trying to build a regression tree with 70 attributes where the business team wants to fix the first two levels namely country and product type.To achieve this,I have two proposals: 1.Build a ...
4
votes
0answers
11k views

Tuning Gradient Boosted Classifier's hyperparametrs and balancing it

I am not sure if it is a correct stack. Maybe I should have put my question into crossvalidated. Nevertheless, I perform following steps to tune the hyperparameters for a gradient boosting model: ...
3
votes
0answers
20 views

Serializing a trained classification model into a set of actionable insights

I'm looking for ways to convert a trained classification model into a list of insights based on the resulting parameters of the model. To make an example, let's assume we trained a decision tree to ...
3
votes
1answer
1k views

How to extract the sample split (values) of decision tree leaves ( terminal nodes) applying h2o library

Sorry for a long story, but it is a long story. :) I am using the h2o library for Python to build a decision tree and to extract the decision rules out of it. I am using some data for training where ...
3
votes
0answers
566 views

Target transformation for tree models

Can anybody explain why/if target variable transformations could help when dealing with tree based models? I've seen this excellent reply which explains quite well why it shouldn't affect if ...
3
votes
0answers
51 views

Is linear regression on the trees of XGBoost (rather than taking their mean) useful/popular?

Given training data $(\underline{x}_1, y_1),...,(\underline{x_N}, y_N)$, one can choose a variety of ensemble method for trees. These algorithms output a set of trees $T_1, ..., T_n$, and then the ...
3
votes
1answer
1k views

how does XGBoost's exact greedy split finding algorithm determine candidate split values for different feature types?

Based on the paper by Chen & Guestrin (2016) "XGBoost: A Scalable Tree Boosting System", XGBoost's "exact split finding algorithm enumerates over all the possible splits on all the features to ...
3
votes
1answer
203 views

Choosing a right algorithm for template-based text generation

I am doing a text generation project -- the task is to basically represent the statistical data in a readable way. The way I decided to go about this is template-based: each data type has a template ...
3
votes
1answer
163 views

Is decision tree regression comparable to locally weighted regression

I am new to decision tree method. For decision tree regression model, does it just fit a piece wise step function over data? When and why would people prefer it over some traditional regression like ...
3
votes
1answer
479 views

Multiclass Classification with Decision Trees: Why do we calculate a score and apply softmax?

I'm trying to figure out why when using decision trees for multi class classification it is common to calculate a score and apply softmax, instead of just taking the averages of the terminal nodes ...
3
votes
1answer
238 views

Does CART algorithm takes into account in the order of the set of attributes?

when using matlab command 'fitctree' for classification purpose, and I change the order of the attributes I do not find the same Tree and thus the same classificaiton error? why? CART algorithm does ...
2
votes
1answer
29 views

Is it possible to do hard-coded decision tree on some variables and random forest / something on the remaining ones?

Is it possible to do hard-coded decision tree on some variables and random forest / something on the remaining ones? The situation seems that for some variables it's possible to draw strong empirical ...
2
votes
0answers
46 views

Creating a new feature from an existing one using decision trees

Is it possible to create a new feature out of two, or more than two existing features using a decision tree? If so, how, and can it produce features with good information value that can better help ...
2
votes
2answers
48 views

KNN efficient implementation

The KNN algorithm is very handy and particularly suited to some of my problems, but I can't find any resources on how to implement it in production. As a comparative example, when I use a neural ...
2
votes
2answers
93 views

How to decode encoded labels in Decision tree classifier

I have some dataset with procurements of organization where actually i'm working. The aim is to find most important features that describe why some processes of purchases is succesful, and why not ...
2
votes
0answers
21 views

Output value of a gradient boosting decision tree node that has just a single example in it

The general gradient boosting algorithm for tree-based classifiers is as follows: Input: training set ${\displaystyle \{(x_{i},y_{i})\}_{i=1}^{n},}{\displaystyle \{(x_{i},y_{i})\}_{i=1}^{n},}$a ...
2
votes
1answer
19 views

Which models implicitly consider interaction between features?

I would like to understand more how different models (NN and RF specifically, but any other as well) consider interaction between features in tabular data? For example, can the model figure out while ...
2
votes
2answers
153 views

Decision Tree Induction using Information Gain and Entropy

I’m trying to build a decision tree algorithm, but I think I misinterpreted how information gain works. Let’s say we have a balanced classification problem. So, the initial entropy should equal 1. ...
2
votes
0answers
888 views

Dealing with categorical variables in Isolation Forest

Isolation Forest is widely used when dealing with outlier/anomaly detection when we have no labels. The theory behind is that making random split at random points and counting how many splits you do ...
2
votes
1answer
128 views

Gini index as a labeling strategy for leaf nodes

Can we use the gini index to assign a class to a leaf node? If yes how? As far as I understand the gini index can only be used as a splitting metric.
2
votes
2answers
46 views

How to decide who to market? Clustering or Decision Tree?

I am working with a dataset that has enough observations and ~ 10 variables, half of the variables are numeric another half of the variables are categorical with 2-3 levels (demographics) one ID ...
2
votes
1answer
104 views

Use cross entropy to create decision tree classifier

Are entropy and cross-entropy the same thing as per basic definition? If there is a difference: Decision tree splits take on entropy or Gini index, can we use cross-entropy to split decision trees? ...
2
votes
0answers
32 views

Decision table reduction

Consider the following Decision table : The following is the reduction process of this table : The above table is the reduced table. But my question why we can't reduce further rule number 3 and 4 ...
2
votes
1answer
180 views

decision tree vs neural network for boolean function

Which structure is more powerful in terms of expressiveness (i.e. it can represent a given Boolean function, accurately) — a single-layer perceptron or a 2-layer decision tree? (There are 10 features)
2
votes
0answers
30 views

How to turn classification tree results into a GIS map

I'm new-ish to machine learning, so this could be a silly question. Apologies if so. The idea is is to predict groundwater occurrence based on a regression tree. This is my conceptual model: Target ...
2
votes
0answers
130 views

Rapidminer and decision tree weights

In Rapidminer, are the decision tree's weights a measure of the "importance" of attributes in the splitting procedure ? If yes, why is useful to know these weights ? Are there better methods to know ...
2
votes
1answer
387 views

Decision trees and Curse of Dimensionality

Since decision tree algorithm splits the training dataset one feature at a time, how the heck is possibly that it suffers from curse of dimensionality ?
2
votes
0answers
466 views

Information Gain vs Gain Ratio in decision trees

I'm studying the decision trees in Data Mining. A weak point of the information gain criterion is that it can lead to an overfitting, a solution can be the use of the gain ratio criterion. May this ...
2
votes
1answer
160 views

Is it feasible to use decision tree algorithms for sensor fault detection?

The gist is me wanting to separate system faults from sensor faults given some dataset from a wireless sensor network using a machine learning algorithm. For instance, if I have some temperature ...
2
votes
1answer
67 views

Machine Learning - Same impurity values

Imagine this example: We see that the attributes color and points have the same value. What attribute should we choose for the ...
2
votes
0answers
560 views

Sales Dataset to determine best model for predicting future sales

We have a set of products in which we are trying to determine which products we should continue to sell, and which products to remove from our inventory. The file contains BOTH historical sales data ...
2
votes
0answers
163 views

How important is lookahead search in decision trees?

I am using random forests, and, in my data, I have a lot of situations where $X_1$ is a bad predictor, $X_2$ is a bad predictor, but the joint distribution would make a good predictor. Say that $X1$, ...
2
votes
0answers
138 views

Parameters for CART tree

I'm using fitctree function from Matlab to fit a CART tree to my dataset (10'000 data points, 20 features). First, how can I exactly prune it and to what level should the tree be pruned? I think I ...
1
vote
1answer
35 views

Decision Trees and Categorical Feature Labelling

I am working on a decision tree model and trying to decide how best to handle categorical features. The features in my dataset are generally high in cardinality and I have found that ordinal labeling ...
1
vote
0answers
9 views

Does anyone know of literature regarding a Neural Net boosted GBM?

For obvious reasons, most GBMs created in the private sector are tree boosted. Occasionally, one might want a linear boosted GBM so that the residual models collapse into a simple linear combination. ...
1
vote
0answers
8 views

Predict pixels in optdigits data set

I'm using this dataset : https://archive.ics.uci.edu/ml/datasets/optical+recognition+of+handwritten+digits a dataset that consists of 65 columns , the last column is the label for 10 classes i.e 0,1,2,...
1
vote
0answers
40 views

Model Tree M5 - Robustness to Data Quality Issues

I am currently investigating the M5 tree algorithm by Quinlan(1992) link here: https://sci2s.ugr.es/keel/pdf/algorithm/congreso/1992-Quinlan-AI.pdf An example of a linear regression model of the ...
1
vote
0answers
10 views

How to implement an oblique decision tree for regression?

There are numerous ways to induce an oblique decision tree in the decision tree induction domain, such as using a support vector machine to determine the best hyper-plane. However, is it possible to ...
1
vote
0answers
21 views

Decision trees vs Oblique decision trees

What are Oblique decision trees ? What are the differences between it and classic decision trees ?
1
vote
0answers
24 views

Information Gain of a Customer ID Attribute

Suppose I have a dataframe like the following ...
1
vote
1answer
44 views

How to model a supervised recommender system with varying data

Suppose there are 2000 movies and a company wants to recommend some movies (for example, at most 5 movies) to each visitor. The objective is to learn how to predict which movie will be selected if a ...
1
vote
0answers
24 views

Extracting rules in regression tree in Python

I don't know if this question is a complicated question or not. I wanna train a regression tree in which in leaves, linear regression is applied to predict. Then, when the tree and linear regression ...
1
vote
0answers
25 views

Calculating the lower and upper bounds forVC-dimension of a decision tree

I have a problem finding the lower and upper bounds of the decision tree. Suppose there is a decision tree with a hypothesis space of depth 2 and an input space with 10 variables (the variables take ...
1
vote
0answers
22 views

Manually building and visualising a decision tree

I am currently working on a decision tree for predicting the average of a binary outcome with a small number of categorical features (think predicting the survival rate in the titanic dataset). I am ...
1
vote
1answer
127 views

Model retraining

I have trained my model with RandomForestRegressor, but now my training data is updated continuously. So I have to train my model with all the train data set i.e past and new data, or can I directly ...