Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
0
votes
1answer
16 views

Using Gini index, how to calculate the probability of correctly classifying a new randomly selected case to the highest probability class? [closed]

I have the following binary Decision Tree: Can you please explain how can I report this tree to a person who only understands probabilities? If ca=1 and cp_4.0=1, what’s the probability of Yes HD?
0
votes
1answer
23 views

Zero-inflated independent feature in tree-based models

What is the best approach to include a zero-inflated continuous independent feature (e.g., 90% of the values are Zero, 10% are >0) in a Tree-based models (DT, random forest, gradient boosting. etc)....
2
votes
1answer
20 views

Can I do bagging method as improvement technique to decision tree in research?

Bagging use decision tree as base classifier. I want to use bagging with decision tree(c4.5) as base as the method that improve decision tree(c4.5) in my research that solve problem overfitting. Is ...
1
vote
1answer
47 views

Why Adaboost SAMME needs f to be estimable?

I am trying to understand the mathematics behind SAMME AdaBoost: At some stage, the paper adds a constraint for f to be estimable: I do not understand why this is ...
1
vote
2answers
25 views

How to find the dependent variables from a dataset?

I am stuck at where how can I get the most dependent variables based on the mean I have this dataset and when I try to: ...
-2
votes
0answers
25 views

Are these precision recall curves possible where single curve intersects each other? [closed]

Someone just showed me this precision-recall curve for decision tree classifiers after feeding scores and targets to scikit learns precision-recall curve module. Is this possible that the single curve ...
1
vote
1answer
15 views

Decision Tree Regressor: domain of the y variable

just wondering about a thing. suppose you fit a Decision Tree Regressor and your training y variable has got a domain that spans ...
2
votes
1answer
37 views

How to handle a regression problem with skewed target and only few high values?

I'm currently tackling a regression problem with skewed target variable (presented below). Naturally, my first idea was to transform the target with natural logarithm as it'll probably help both ...
1
vote
1answer
44 views

How could a neural network classifer for multilclass problem classify only in one class when a decision tree is more balanced and accurate?

I want to create a classifier for a data frame that has four classes. Each line can only have one class. I have two predictive models: a neural network and a tree classifier. But they put everyone in ...
0
votes
1answer
8k views

How to avoid memory error with Pandas pd.read_csv method call with GridSearchCV usage for DecisionTreeRegressor model?

I have been implementing a DecisionTreeRegressor model in Anaconda environment with a data set sourced from a 20 million row, 12-dimensional CSV file. I could get the chunks off of the data set with ...
0
votes
0answers
16 views

Confusion matrix and Features Regularization applied to Decision Tree [closed]

Ok, As far as I understand we can use the f-score to select the best algorithm for our classification. We can set different values of threshold and see which maximizes the f-score to select the best ...
1
vote
1answer
36 views

Interpreting decision tree results after target encoding

I am not sure how to interpret the results of my decision tree after I had used target encoding, could someone clarify? The example below doesn't need target encoding just for explanation of my ...
1
vote
1answer
217 views

Isolation Forest Score Function Theory

I am currently reading this paper on isolation forests. In the section about the score function, they mention the following. For context, $h(x)$ is definded as the path length of a data point ...
0
votes
0answers
13 views

Collapse categorical variable to reduce levels using a decision tree

I am using zip codes as an independent variable as part of a binary classification problem. Naturally, this feature has many different levels (around 2,000), so I was wondering if there is a ...
0
votes
1answer
26 views

NN Making Poor Averaging Fit whilst LGBM Regressor Fits Perfectly

I have a simple toy dataset for which the features have been encoded using a Encoder-Decoder NN. I am using the hidden feature vector from the Encoder as the X input for training a 1-step lookahead ...
1
vote
1answer
52 views

splitting mechanism with one hot encoded variables (tree based/boosting)

I am using xgboost and have a categorical unordered feature with 25 levels. So when i apply one hot encoding i have 25 columns. This introduces alot of sparsity. Even more unusual, my feature ...
-1
votes
1answer
27 views

How to identify Overfitting in RandomForestClassifier?

Im building a sentiment classification model using RandomForestClassifier. I got the training accuracy of 99.65 & cross-validation( RepeatedStratifiedKFold-5 folds) accuracy of 97.29. I used f1 ...
3
votes
1answer
228 views

Does CART algorithm takes into account in the order of the set of attributes?

when using matlab command 'fitctree' for classification purpose, and I change the order of the attributes I do not find the same Tree and thus the same classificaiton error? why? CART algorithm does ...
1
vote
1answer
38 views

Traditional ML Model or Deep Learning for ~200-300 samples?

Good morning all! I'm working on a resume parser that is integrated with an RPA (robotic process automation) platform. The robot has OCR to extract text from a PDF resume, and it supplies the ...
-1
votes
0answers
8 views

Build Tree is not working [migrated]

I am very new to orange. That said, I have written a code snippet: ...
4
votes
3answers
363 views

Which models can handle null values?

Unfortunately trying to google or research null values in machine learning always brings up pages trying to teach you how to impute the values instead, but I'm trying to find models that can handle ...
2
votes
1answer
124 views

Is it feasible to use decision tree algorithms for sensor fault detection?

The gist is me wanting to separate system faults from sensor faults given some dataset from a wireless sensor network using a machine learning algorithm. For instance, if I have some temperature ...
3
votes
1answer
139 views

Choosing a right algorithm for template-based text generation

I am doing a text generation project -- the task is to basically represent the statistical data in a readable way. The way I decided to go about this is template-based: each data type has a template ...
1
vote
2answers
149 views

how does splitting occur at a node in a decision-tree with non-categorical data?

According to a website (:http://dataaspirant.com/2017/01/30/how-decision-tree-algorithm-works/) , these values are chosen randomly: I don't think this is the case with any optimized way of creating ...
0
votes
1answer
1k views

How to build a symptom checker and medical diagnose chat bot

I want to build a chatbot which can diagnose an illness depending upon the symptoms which are given to it. I want to show you an example scenario of how it should work; Application - A , User - U ...
1
vote
1answer
125 views

Decision Tree: Efficient splitting of nodes, minimize number of gini evaluations

I have a dataset specific problem where i need to use a splitting function other than gini_index. This requires me to re-write a decision tree from scratch. I have a working model, but itis highly ...
4
votes
2answers
2k views

fix first two levels of decision tree?

I am trying to build a regression tree with 70 attributes where the business team wants to fix the first two levels namely country and product type.To achieve this,I have two proposals: 1.Build a ...
2
votes
1answer
25 views

XGBoost Tree 'starting feature break'

I am fairly new to learning the XGBoost algorithm and had a question about how the algorithm knows which feature to break the tree on first. Here is my understanding (and please correct me if I'm ...
1
vote
1answer
65 views

Decision Trees change result at every run, how can I trust of my results?

Given a database, I split the data in train and test. I want to use a decision-tree classifier (sklearn) for a binary classification problem. Considering I already found the best parameters for my ...
0
votes
0answers
28 views

Classification and decision trees for beginners

I'm working with a social science dataset about the academic performance of university students. There are many variables on the dataset (Gender, Vital records, education level of parents, attendance ...
1
vote
0answers
27 views

Prediction in CART Decision Trees

I was studying the algorithm of CART (classification and regression trees), but the formula of the prediction is irritating me. First we have the following definition: Let $X:={x_1,...,x_N} \subset \...
1
vote
1answer
169 views

Details on soft decision trees

In Distilling a Neural Network Into a Soft Decision Tree the authors mention exponentially decaying running average of the actual probabilities with a time window but provide no formula for it. ...
3
votes
1answer
404 views

Multiclass Classification with Decision Trees: Why do we calculate a score and apply softmax?

I'm trying to figure out why when using decision trees for multi class classification it is common to calculate a score and apply softmax, instead of just taking the averages of the terminal nodes ...
16
votes
4answers
28k views

Decision tree vs. KNN

In which cases is it better to use a Decision tree and other cases a KNN? Why use one of them in certain cases? And the other in different cases? (By looking at its functionality, not at the ...
4
votes
5answers
6k views

Are Decision Trees Robust to Outliers

I read that decision trees (I am using scikit-learn's classifier) are robust to outlier. Does that mean that I will not have any side-effect if I choose not to remove my outliers?
1
vote
1answer
27 views

How does the construction of a decision tree differ for different optimization metrics?

I understand how a decision tree is constructed (in the ID3 algorithm) using criterion like entropy, gini index, variance reduction. But the formulae for these criteria do not care about optimization ...
3
votes
1answer
625 views

Not able to interpret decision tree when using class_weights

I'm working with an imbalanced dataset. I'm using a decision tree (scikit-learn) to build a model. For explaining my problem I've taken iris dataset. When I'm setting ...
1
vote
1answer
66 views

Misclassification Rate for Random Forest Plateauing too Early

Using R, I have created 5 different random forest models using 5 different numbers of trees (3,10,30,100,300). My intention was to compute the misclassification rates of each of these models and plot ...
1
vote
1answer
26 views

Number of leaves for lightgbm is smaller than categories in one feature

I was looking at a notebook someone posted for a Kaggle competition. They use lightgbm with the number of leaves set to 40. If I understand right, that's setting a limit on the size of the weak ...
4
votes
1answer
86 views

How do we decide between XGBoost, RandomForest and Decision tree?

What do we take into consideration while deciding which technique should be used when dealing with a particular dataset? I understand that there isn't any hard and fast rule to this. Do we use XGBoost ...
1
vote
1answer
90 views

decision -tree regression to avoid multicollinearity for regression model?

I read in comments a recommendation for decision tree´s instead of linear models like neural network, when the dataset has many correlated features. Because to avoid multicollinearity. A similar ...
5
votes
2answers
995 views

LightGBM - Why Exclusive Feature Bundling (EFB)?

I'm currently studying GBDT and started reading LightGBM's research paper. In section 4. they explain the Exclusive Feature Bundling algorithm, which aims at reducing the number of features by ...
3
votes
2answers
85 views

Is Label Encoding with arbitrary numbers ever useful at all?

From what I read online, there seems to be some confusion regarding the taxonomy and the terms used, so to avoid misunderstanding I'm going to define them here: Label Encoding - encoding a nominal ...
6
votes
1answer
3k views

Is it necessary to normalise data for XGBoost?

MinMaxScaler in scikit_learn is used for data normalization (a.k.a feature scaling). Data normalisation is not necessary for decision trees. Since XGBoost is based on decision trees, is it necessary ...
2
votes
0answers
17 views

Output value of a gradient boosting decision tree node that has just a single example in it

The general gradient boosting algorithm for tree-based classifiers is as follows: Input: training set ${\displaystyle \{(x_{i},y_{i})\}_{i=1}^{n},}{\displaystyle \{(x_{i},y_{i})\}_{i=1}^{n},}$a ...
2
votes
1answer
794 views

How to extract the sample split (values) of decision tree leaves ( terminal nodes) applying h2o library

Sorry for a long story, but it is a long story. :) I am using h2o library for python to build a decision tree and to extract a decision rules out of it. I use some data for training where labels get ...
2
votes
1answer
64 views

SKLearn DT regressor - good enough score?

What constitutes as a "good enough" score for a Decision Tree Regressor? The .score() function gives us a general score about our model. This can be 1 if the model ...

1
2 3 4 5
11