Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

Custom Decision Function for Custom Outlier Detection Algorithm

I have built a custom algorithm for semi-supervised anomaly detection and here is my output example as following with probability threshold set to 0.05 and 1 = outlier, 0 = inlier: ...
1
vote
1answer
29 views

Decision Trees change result at every run, how can I trust of my results?

Given a database, I split the data in train and test. I want to use a decision-tree classifier (sklearn) for a binary classification problem. Considering I already found the best parameters for my ...
1
vote
0answers
22 views

Extract features from Decision tree leaf nodes

Recently came across a coursera course on "How to win Kaggle competitions" where they explain how we can engineer a categorical feature from each leaf node of the decision tree. [Video Link][1] I ...
1
vote
1answer
21 views

How does the construction of a decision tree differ for different optimization metrics?

I understand how a decision tree is constructed (in the ID3 algorithm) using criterion like entropy, gini index, variance reduction. But the formulae for these criteria do not care about optimization ...
0
votes
1answer
19 views

Can one property name be used twice in the same branch of a DecisionTreeRegressor?

I am using this dataset for the analysis (Generated using make_regression of sklearn library) I was trying to learn the DecisionTreeRegression algorithm of sklearn library. I used the following code ...
3
votes
0answers
17 views

Serializing a trained classification model into a set of actionable insights

I'm looking for ways to convert a trained classification model into a list of insights based on the resulting parameters of the model. To make an example, let's assume we trained a decision tree to ...
2
votes
0answers
23 views

Random forest mode scoring

We are using random forest algorithm but having some trouble understanding the scoring method it uses. take for example the following CM of the test set: ...
5
votes
2answers
167 views

XGBOOST - different result between train_test_split and manually splitting

I am trying to train XGBOOST model. X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=43, stratify=y) when I'm using ...
1
vote
1answer
39 views

Interpretable xgboost - Calculate cover feature importance

When trying to interpret the results of a gradient boosting (or any decision tree) one can plot the feature importance. There are same parameters in the xgb api sucha as: weight, gain, cover, ...
0
votes
0answers
27 views

Multiclass XGBoost train with num classes = 2

I have a tagged csv file with 5 calsses. I accidentally trained am XGBOOST model with this input but forgot to change the num_classes to 5, but instead it was still 2. The model I received seems to ...
0
votes
0answers
11 views

How to calculate “minimal confidence for the correct label”

In RapidMiner one of the decision tree performance measures is called the margin. The margin is defined as "minimal confidence for the correct label". Can someone explain to me what it means and how ...
1
vote
2answers
25 views

Python and GridSearchCV how to eliminate input contains NaN error when using cross validation and decision tree classifier?

I am trying to do cross validation on Decision tree classifier for kaggle's titanic dataset. The first step after cleaning data is to split into train and test sets: ...
1
vote
1answer
41 views

Correlation based Feature Selection vs Feature Engineering

I'm a bit confused about the superiority of Feature Selection over Feature Engineering or vice versa. Let's say I just want to get the best possible performance on a couple of models like a neural ...
1
vote
1answer
22 views

Which decision tree model is used in “standard” random forest?

Is that CART? Why not using C5.0 tree? A perhaps more general question: Since C5.0 tree frequently have better performance than CART, why people still use CART to build random forest (or people ...
2
votes
1answer
27 views

How do I know the best pruning criteria for decision trees?

Right now,I am working on decision trees on python,how do I know what would be the best pruning criteria based on my data?
5
votes
1answer
104 views

Why gradient boosting uses sampling without replacement?

In Random Forest each tree is built selecting a sample with replacement (bootstrap). And I assumed that Gradient Boosting's trees were selected with the same sampling technique. (@BenReiniger ...
0
votes
1answer
10 views

Number of leaves for lightgbm is smaller than categories in one feature

I was looking at a notebook someone posted for a Kaggle competition. They use lightgbm with the number of leaves set to 40. If I understand right, that's setting a limit on the size of the weak ...
1
vote
0answers
16 views

How do decision tree works for feature selection?

I have a dataframe with a feature selection problem. I want to get the variables explaining the variance within each segment of the following dataset: ...
1
vote
0answers
22 views

CP plot in decision tree showing upward trend only

In cp plot for decision tree, we expect the error to drop down with size of the tree (or form an elbow). This helps to select right cp value. But in my graph the relative error is continuously ...
2
votes
1answer
42 views

Decision Tree on Unbalanced Data

While running decision tree, i have unbalanced data. The data balance is 93%(Class 0) to 7% (Class 1). Now when i am plotting decision tree to understand the factors contributing to class 1 then in ...
0
votes
1answer
60 views

Decision tree in sentiment analysis

Creating a classifier to do "Sentiment Analysis" can be done with several algorithms like SVM, KNN, Neural networks,Decision tree... and lately i have read about Decision tree and how it works and i ...
2
votes
1answer
23 views

Classification algorithm with multiple output for a set of features

I want to build a classification algorithm that will predict multiple values for a set of features. For instance, lets say I have a customer demographic data like Income, age, sex, city and I want to ...
3
votes
1answer
59 views

Which models can handle null values?

Unfortunately trying to google or research null values in machine learning always brings up pages trying to teach you how to impute the values instead, but I'm trying to find models that can handle ...
5
votes
1answer
43 views

Search for hyperparameters whith different features using Random Forest

I have a dataset in which I would like to perform a classification model, so I have decided to use Random Forest. The number of features that I have is approximately 200 and I would like to test which ...
0
votes
0answers
17 views

Entropy vs Gini Gain in decision tree [duplicate]

In decision tree when to use entropy and when gini gain for growing the tree?
3
votes
1answer
48 views

Make a random forest estimator the exact same of a decision tree

The idea is to make one of the trees of a Random Forest, to be built exactly equal to a Decision Tree. First, we load all libraries, fit a decision tree and plot it. ...
1
vote
1answer
54 views

Decision tree regression: Polynomials unnecessary?

I am testing out different models for a regression task. When using OLS, Ridge and Lasso, I use different polynomial degrees of the explanatory variables. Example: For two variables x and y, degree 2 ...
0
votes
0answers
16 views

CN2 vs. decision trees

I would like to better understand the fundamental differences between CN2 rules and decision trees, especially when CN2 rules are of the kind "decision list" (with rule ordering). E.g. when applied to ...
0
votes
0answers
27 views

gui for avl tree

Implement a GUI for AVL Tree for int data type using JavaFX. The GUI should provide facility to add, remove and find data elements and also traversal (in-order, pre-order, post-order, level-order) of ...
1
vote
0answers
23 views

Use cross entropy to create decision tree classifier

Are entropy and cross-entropy the same thing as per basic definition? If there is a difference: Decision tree splits take on entropy or Gini index, can we use cross-entropy to split decision trees? ...
8
votes
1answer
321 views

Why continuous features are more important than categorical features in decision tree models?

I have both categorical and continuous features in my prediction model and want to select (and rank) most important features. I have converted all categorical variables into dummy variables using one ...
0
votes
1answer
47 views

Feature importance and deriving rules using tree based classification models

I have a dataset where I have categorical and continuous values with targets 0/1 (binary classification task). Since I need to find patterns and relationships in the occurrence of the event or target, ...
2
votes
0answers
19 views

Decision table reduction

Consider the following Decision table : The following is the reduction process of this table : The above table is the reduced table. But my question why we can't reduce further rule number 3 and 4 ...
1
vote
1answer
63 views

How is the 'feature_importance_' value calculated (in sklearn modules) for each variable in a random forest regressor?

I have 9000 sample, with five features, and one output variable (all are numerical, continuous values). I used random forest regression method using scikit modules. ...
1
vote
1answer
34 views

what are the steps in adaboosting?

I went through adaboost tutorial and below are my simplified understanding: Sample weight of equal value is given to all sample in dataset. Stumps are created which uses only one feature from data ...
2
votes
2answers
65 views

how to see decision tree when running in anaconda?

In Jupyter displaying the DT is done as follows: # Display in jupyter notebook from IPython.display import Image Image(filename = 'tree.png') How to see DT ...
4
votes
1answer
195 views

How are samples selected from training data in Xgboost

In Random Forest, each tree is not fed with the full batch of training data, only a sample. How does this work for Xgboost? If this sampling happens as well, how does it work for this ML algorithm?
0
votes
1answer
304 views

What is the difference between Freidman mse and mse?

The original question was posted on StackOverflow. However, taking into account the recommendations of Sergey Bushmanov, I'm providing an answer through this medium.
5
votes
1answer
660 views

Is there any difference between a weak learner and a weak classifier?

I am reading about Gradient Boosting, AdaBoost etc. I have found the following two concepts weak learner and weak classifier. Are they the same? If there is any difference what is it?
3
votes
1answer
52 views

Is there a real C4.5 implementation in Python ? (handling missing value)

To my understanding, C4.5 comes with 4 improvements compared to ID3: Handling missing values in both training data and "test" data, Handling continuous data Handling costs on attributes. The pruning ...
1
vote
0answers
34 views

Sklearn Decision Tree as weak learner in Adaboost not working properly

I'm trying to implement Adaboost algorithm with sklearn decision tree as the Weak Learner - at each step I want to choose one feature with one threshold to classify all samples. I have 1400 long ...
6
votes
5answers
644 views

Decision tree with final decision being a linear regression

Question: I want to implement a decision tree with each leaf being a linear regression, does such a model exist (preferable in sklearn)? Example case 1: Mockup data is generated using the formula: <...
1
vote
3answers
67 views

Will the MAE of testing data always be higher than MAE of training data?

On the Kaggle Course Page the chart below shows that MAE of testing data is always higher than MAE of training data. Why is this the case? Is it only limited to DecisionTreeRegressor model? Or the ...
3
votes
2answers
46 views

(Newbie) Decision Tree Classifier Splitting precedure

I have a dataset with 4 categorical features (Cholesterol, Systolic Blood pressure, diastolic blood pressure, and smoking rate). I use a decision tree classifier to find the probability of stroke. I ...
1
vote
1answer
27 views

Why there are no tutorials on calculating Gini impurity in a three-valued categorical feature variables?

Trying to learn here so please go easy on me if I asked dumb questions. As the title says, I was searching for a tutorial that calculates the Gini in a CART algorithm for a three-valued feature ...
4
votes
2answers
73 views

Why Decision Tree Classifier is not working with categorical value?

I am learning my way through this, so please be easy on me if you find any mistakes, I could really use a professional opinion here. Thx. I am trying to model a Decision Tree Classifier as part of an ...
3
votes
1answer
41 views

Decision Tree Classifier to classify values based on values of other columns

I have data with multiple labels, for example My X set is fromt second to third column, and I want to classify either first column or the last column, so I made my Y the last column. The goal is so ...
0
votes
1answer
106 views

How to select features which have non zero importance using SKLearn's Decision Trees Classifier?

I need to discard all the features which have zero importance and keep only those features which have non zero importance while implementing DecisionTreesClassifier. The feature importance here is ...
1
vote
1answer
21 views

DecisionTreeClassifier Integer Conditions, Integer Outcome Variable [closed]

Vague condition: "NumGoals >= 1.23" Preferred condition: "NumGoals > 1". Switched normalization off. Code: ...
1
vote
2answers
42 views

Can I force DecisionTreeClassifier to use integer conditions when the variable is integer?

I'm trying to visualize a decision tree in python for the purpose of explainability. I noticed that a condition like "NumGoals >= 1.23" could be quite vague for the user and I would much rather to see ...

1
2 3 4 5
10