Skip to main content

Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
2 votes
1 answer
83 views

How do I interpret probability results in conjunction with my known precision/accuracy/recall scores?

I have a Random Forest Classifier (trained with sklearn) modeling a binary data set. Here's what the configuration looks like (I've tuned it for precision intentionally): ...
5 votes
2 answers
667 views

Loss for ordered multi class data in classification

Assume data which is labeled $y_i \in \left\{ 1, 2, 3, \ldots, 9, 10\right\}$. Assume the labels are ordered, namely, given $y_i = 10$ to estimate $\hat{y}_{i} = 1$ is much worse than $\hat{y}_{i} = ...
4 votes
2 answers
4k views

Fix first two levels of decision tree?

I am trying to build a regression tree with 70 attributes where the business team wants to fix the first two levels namely country and product type. To achieve this, I have two proposals: Build a ...
2 votes
1 answer
51 views

Classifying short strings of text with additional context

I have a list of short strings each identifying a city. Misspellings are very common. The example below shows some of these short strings, along with the correct city they're supposed to match. ...
0 votes
1 answer
25 views

In natural language processing, what is the name for the technique in which a sentence is modeled as a tree in order to generate simpler sentences?

In natural language processing, there are times when we model a complex and/or compound sentence as a tree (or hierarchy) of simpler sentences. The tree-model (hierarchical model) can help us ...
0 votes
2 answers
3k views

What is the max number of leaf nodes in a classifcation decision tree?

Let's assume that we have n observations and p predictors and we have in a n>>p situation. All predictors are binary. What is the max number of leaf nodes that we can have in the tree? and what ...
5 votes
1 answer
679 views

Multi-target regression tree with additional constraint

I have a regression problem where I need to predict three dependent variables ($y$) based on a set of independent variables ($x$): $$ (y_1,y_2,y_3) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \...
1 vote
1 answer
259 views

Decision trees for anomaly detection

Problem From what I understand, a common method in anomaly detection consists in building a predictive model trained on non-anomalous training data, and perform anomaly detection using the error of ...
2 votes
1 answer
227 views

Is it feasible to use decision tree algorithms for sensor fault detection?

The gist is me wanting to separate system faults from sensor faults given some dataset from a wireless sensor network using a machine learning algorithm. For instance, if I have some temperature ...
3 votes
1 answer
270 views

Is decision tree regression comparable to locally weighted regression

I am new to decision tree method. For decision tree regression model, does it just fit a piece wise step function over data? When and why would people prefer it over some traditional regression like ...
3 votes
1 answer
276 views

Does CART algorithm takes into account in the order of the set of attributes?

when using matlab command 'fitctree' for classification purpose, and I change the order of the attributes I do not find the same Tree and thus the same classificaiton error? why? CART algorithm does ...
0 votes
0 answers
13 views

Latest Tree-based models

What are the latest Tree-based models that are used in machine learning? Tell the new models except the old ones such as the Decision tree, Random Forest, Gradient Boosting, LightGBM, XGBoost, and ...
0 votes
2 answers
185 views

Should I resample my dataset?

The dataset that I have is some text data consisting of path names. I am using TF-IDF vectorizer and decision trees. The classes in my dataset are severely imbalanced. There are a few big classes with ...
11 votes
9 answers
48k views

I got 100% accuracy on my test set,is there something wrong?

I got 100% accuracy on my test set when trained using decision tree algorithm.but only got 85% accuracy on random forest Is there something wrong with my model or is decision tree best suited for the ...
4 votes
1 answer
253 views

How important is lookahead search in decision trees?

I am using random forests, and, in my data, I have a lot of situations where $X_1$ is a bad predictor, $X_2$ is a bad predictor, but the joint distribution would make a good predictor. Say that $X1$, ...
1 vote
1 answer
26 views

Average training instances sampled with bagging

The book Hands-On Machine Learning has a section on Out-of-Bag Evaluation related to Decision Trees, where it's stated that, By default a BaggingClassifier samples m training instances with ...
3 votes
1 answer
537 views

Choosing a right algorithm for template-based text generation

I am doing a text generation project -- the task is to basically represent the statistical data in a readable way. The way I decided to go about this is template-based: each data type has a template ...
0 votes
0 answers
12 views

I want to create a system for classifying bone fractures What pre-processing steps can I use to process images?

I want to know where I should put the image preprocessing code in the decision tree code How to extract features from images and classify them
2 votes
1 answer
2k views

How to extract the sample split (values) of decision tree leaves ( terminal nodes) applying h2o library

Sorry for a long story, but it is a long story. :) I am using the h2o library for Python to build a decision tree and to extract the decision rules out of it. I am using some data for training where ...
0 votes
1 answer
339 views

Low accuracy on the test set

I have a dataset with 16 features and 32 class labels, which shows the following behavior: Neural network classification: high accuracy on train 100%, but low accuracy on the test set 3% (almost like ...
3 votes
1 answer
199 views

help interpreting training/validation curves for classification tree

I'm developing a binary classification tree and having some touble interpreting my training/validation curves. I used the CART algorithm with information gain as my splitting criterion. The training ...
8 votes
1 answer
4k views

Decision Trees - C4.5 vs CART - rule sets

When I read the scikit-learn user manual about Decision Trees, they mentioned that CART (Classification and Regression Trees) is very similar to C4.5, but it differs in that it supports numerical ...
0 votes
1 answer
60 views

Give more weight to features based on distribution plot

I have a task to predict a binary variable purchase, their dataset is strongly imbalanced (10:100) and the models I have tried so far (mostly ensemble) fail. In ...
0 votes
2 answers
99 views

Random Forest plot standardized

For a data science project, I first used a standardized scaler on data in python, ran random forest then plotted the tree. However, the values of the decisions are in their standardized form. How do I ...
0 votes
1 answer
125 views

Optimizing decision tree

I have a question regarding the technique/technology which could be applied for the issue: Suppose I have a rule-based tree or decision tree which predicts a variable Y based on variables A,B,C. This ...
0 votes
1 answer
3k views

Decision tree vs logistic regression feature importances

I have trained Logistic regression and decision tree in skearn on the same standardized dataset (binary classification). Top important coefficients for the decision tree are (sorted by ...
1 vote
1 answer
193 views

Upper bound on size of sample set for decision trees

Say I have an instance space with 4 features and I know that a decision tree with 8 nodes can represent the target function I want to learn. I want to give an upper bound on the size of the sample set ...
1 vote
3 answers
693 views

Advantages and disadvantages of using classification tree

I was working on a project and was trying to validate my decisions. I wondered why would I want to use a decision tree over more powerful algorithms like random forest or Gradient boosting machine ...
1 vote
1 answer
114 views

DART algorithm implementation. Converting mathematical notation to pseudocode

I am learning how DART algorithm (https://arxiv.org/abs/1505.01866) works and I want to implement it in C# I have the algorithm's description in mathematical notation and I don't understand most of it....
1 vote
1 answer
58 views

How to model a supervised recommender system with varying data

Suppose there are 2000 movies and a company wants to recommend some movies (for example, at most 5 movies) to each visitor. The objective is to learn how to predict which movie will be selected if a ...
2 votes
1 answer
1k views

Giving more weight to a particular feature in scikit-learn decision trees

I have a model that I train on same data, but i want a feature to have a stronger weight. Say I have three features: Car manufacturer's name Price Top speed ...
0 votes
0 answers
57 views

Decision Tree prediction for the fail reason

In my experiment, I used Decision Trees to predict whether participants will pass or fail, and I will provide feedback to them based on the reason for their failure. The Decision Tree includes three ...
0 votes
2 answers
66 views

how to deal if input feature are all categorical but target feature is discrete numerical, it is giving exact numerical for known data

the following data is to detect malnutrition among children under age 5 and the value_r is the percentage estimate of wasting among the population. should I apply the decision tree as an entirely ...
2 votes
2 answers
617 views

Decision Tree Induction using Information Gain and Entropy

I’m trying to build a decision tree algorithm, but I think I misinterpreted how information gain works. Let’s say we have a balanced classification problem. So, the initial entropy should equal 1. ...
3 votes
3 answers
264 views

CART algorithm (Classification and regression trees) question

We fit a full classification tree model $T_k$ of given depth $k$ to data using the CART algorithm, and prune the tree by finding $E(k, \alpha) = min_{T\subset Tk} Err(T) + \alpha |T|$. Here, $Err(T)$ ...
0 votes
1 answer
74 views

Which decision tree algorithm does H2O use?

Does H2O's plain random forest use CART, C4.5, 5.0, or something else? I cannot find this information. sklearn's docs say they use a modified version of CART, and I assume H2O also uses something like ...
2 votes
2 answers
646 views

What can weka do that python and sklearn can't?

I would like to build a variety of classification and regression decision trees. My use case focuses on extraction and communication of decision rules. Previously weka was used in my organisation for ...
2 votes
1 answer
2k views

Decision tree ordering

I am interested in finding out how decision trees chose the order in which they split. I understand that splitting is based in information gain. The attribute with the lowest information gain is ...
0 votes
1 answer
167 views

how to compute the possible number of splits in decision tree?

In the following dataset, if we want to include just two variables, STORE and PctDiscMM, in a classification tree model, what is the possible number of first splits? ...
4 votes
2 answers
743 views

Multiclass Classification with Decision Trees: Why do we calculate a score and apply softmax?

I'm trying to figure out why when using decision trees for multi class classification it is common to calculate a score and apply softmax, instead of just taking the averages of the terminal nodes ...
3 votes
1 answer
152 views

Output value of a gradient boosting decision tree node that has just a single example in it

The general gradient boosting algorithm for tree-based classifiers is as follows: Input: training set $\{(x_{i},y_{i})\}_{i=1}^{n}$, a differentiable loss function $L(y,F(x))$, and a number of ...
0 votes
0 answers
23 views

How to visualize a decision tree classifier?

I'm using ID3 algorithm to build a classifier and was wondering if there is any way to visualize the decision tree that the algorithm builds. This is my code for a decision tree in Python: ...
5 votes
2 answers
1k views

Decision Trees Should We Discard Low Importance Features?

I just started to work with feature selection. Let's say I have a decision tree model. I get its feature importances by tree.feature_importances_. In my model out ...
1 vote
2 answers
215 views

separate decision tree for categorical feature values

Given either, different decision trees each based on a particular feature value (like separate models for each male and female) or a single decision tree, should both give the same result?
2 votes
0 answers
29 views

What are the analogies between decision trees and neural trees?

How can I draw analogies between decision trees and neural trees? For example, how are splitting thresholds analogous between these models, and how can paths in a neural tree be represented in a ...
0 votes
1 answer
141 views

How does LGBM make a prediction?

We are currently trying to figure out how LGBM creates its trees and how predictions are made afterwards. In my current understanding, it works as follows: Multiple "weak learners" are ...
1 vote
3 answers
199 views

How to find the dependent variables from a dataset?

I am stuck at where how can I get the most dependent variables based on the mean I have this dataset and when I try to: ...
1 vote
1 answer
352 views

Model retraining

I trained my model with RandomForestRegressor(), but now my training data is updated continuously. So I have to train my model with all the train data set i.e past ...
0 votes
2 answers
160 views

How to create classification decision trees on a dataset that has both numerical and categorical variables?

I am quite new to Data Science and learning things hands-on in the job. I am a fraud analyst and my job is to predict whether an application is fraudulent or not based on data. Before moving on to ...
2 votes
2 answers
1k views

How to decode encoded labels in Decision tree classifier

I have some dataset with procurements of organization where actually i'm working. The aim is to find most important features that describe why some processes of purchases is succesful, and why not ...

1
2 3 4 5
16