Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
1
vote
1answer
23 views

Correlation based Feature Selection vs Feature Engineering

I'm a bit confused about the superiority of Feature Selection over Feature Engineering or vice versa. Let's say I just want to get the best possible performance on a couple of models like a neural ...
1
vote
1answer
20 views

Which decision tree model is used in “standard” random forest?

Is that CART? Why not using C5.0 tree? A perhaps more general question: Since C5.0 tree frequently have better performance than CART, why people still use CART to build random forest (or people ...
2
votes
1answer
25 views

How do I know the best pruning criteria for decision trees?

Right now,I am working on decision trees on python,how do I know what would be the best pruning criteria based on my data?
4
votes
0answers
45 views

Why gradient boosting uses sampling without replacement?

In Random Forest each tree is built selecting a sample with replacement (bootstrap). And I assumed that Gradient Boosting's trees were selected with the same sampling technique. (@BenReiniger ...
0
votes
0answers
5 views

Number of leaves for lightgbm is smaller than categories in one feature

I was looking at a notebook someone posted for a Kaggle competition. They use lightgbm with the number of leaves set to 40. If I understand right, that's setting a limit on the size of the weak ...
-1
votes
0answers
41 views

How to adapt my decision tree for binary classification to a decision tree for multi-label classification?

I would like to adapt a decision tree algorithm that works for two labels: True/False to one that can decide using multiple labels. ...
1
vote
0answers
16 views

How do decision tree works for feature selection?

I have a dataframe with a feature selection problem. I want to get the variables explaining the variance within each segment of the following dataset: ...
1
vote
0answers
20 views

CP plot in decision tree showing upward trend only

In cp plot for decision tree, we expect the error to drop down with size of the tree (or form an elbow). This helps to select right cp value. But in my graph the relative error is continuously ...
1
vote
1answer
37 views

Decision Tree on Unbalanced Data

While running decision tree, i have unbalanced data. The data balance is 93%(Class 0) to 7% (Class 1). Now when i am plotting decision tree to understand the factors contributing to class 1 then in ...
1
vote
1answer
19 views

Decision tree in sentiment analysis

Creating a classifier to do "Sentiment Analysis" can be done with several algorithms like SVM, KNN, Neural networks,Decision tree... and lately i have read about Decision tree and how it works and i ...
2
votes
1answer
16 views

Classification algorithm with multiple output for a set of features

I want to build a classification algorithm that will predict multiple values for a set of features. For instance, lets say I have a customer demographic data like Income, age, sex, city and I want to ...
3
votes
1answer
26 views

Which models can handle null values?

Unfortunately trying to google or research null values in machine learning always brings up pages trying to teach you how to impute the values instead, but I'm trying to find models that can handle ...
5
votes
1answer
40 views

Search for hyperparameters whith different features using Random Forest

I have a dataset in which I would like to perform a classification model, so I have decided to use Random Forest. The number of features that I have is approximately 200 and I would like to test which ...
0
votes
0answers
16 views

Entropy vs Gini Gain in decision tree [duplicate]

In decision tree when to use entropy and when gini gain for growing the tree?
3
votes
1answer
44 views

Make a random forest estimator the exact same of a decision tree

The idea is to make one of the trees of a Random Forest, to be built exactly equal to a Decision Tree. First, we load all libraries, fit a decision tree and plot it. ...
1
vote
1answer
48 views

Decision tree regression: Polynomials unnecessary?

I am testing out different models for a regression task. When using OLS, Ridge and Lasso, I use different polynomial degrees of the explanatory variables. Example: For two variables x and y, degree 2 ...
0
votes
0answers
12 views

CN2 vs. decision trees

I would like to better understand the fundamental differences between CN2 rules and decision trees, especially when CN2 rules are of the kind "decision list" (with rule ordering). E.g. when applied to ...
0
votes
0answers
23 views

gui for avl tree

Implement a GUI for AVL Tree for int data type using JavaFX. The GUI should provide facility to add, remove and find data elements and also traversal (in-order, pre-order, post-order, level-order) of ...
1
vote
0answers
17 views

Use cross entropy to create decision tree classifier

Are entropy and cross-entropy the same thing as per basic definition? If there is a difference: Decision tree splits take on entropy or Gini index, can we use cross-entropy to split decision trees? ...
8
votes
1answer
276 views

Why continuous features are more important than categorical features in decision tree models?

I have both categorical and continuous features in my prediction model and want to select (and rank) most important features. I have converted all categorical variables into dummy variables using one ...
0
votes
1answer
45 views

Feature importance and deriving rules using tree based classification models

I have a dataset where I have categorical and continuous values with targets 0/1 (binary classification task). Since I need to find patterns and relationships in the occurrence of the event or target, ...
2
votes
0answers
19 views

Decision table reduction

Consider the following Decision table : The following is the reduction process of this table : The above table is the reduced table. But my question why we can't reduce further rule number 3 and 4 ...
1
vote
1answer
53 views

How is the 'feature_importance_' value calculated (in sklearn modules) for each variable in a random forest regressor?

I have 9000 sample, with five features, and one output variable (all are numerical, continuous values). I used random forest regression method using scikit modules. ...
1
vote
1answer
31 views

what are the steps in adaboosting?

I went through adaboost tutorial and below are my simplified understanding: Sample weight of equal value is given to all sample in dataset. Stumps are created which uses only one feature from data ...
2
votes
2answers
55 views

how to see decision tree when running in anaconda?

In Jupyter displaying the DT is done as follows: # Display in jupyter notebook from IPython.display import Image Image(filename = 'tree.png') How to see DT ...
4
votes
1answer
145 views

How are samples selected from training data in Xgboost

In Random Forest, each tree is not fed with the full batch of training data, only a sample. How does this work for Xgboost? If this sampling happens as well, how does it work for this ML algorithm?
0
votes
1answer
147 views

What is the difference between Freidman mse and mse?

The original question was posted on StackOverflow. However, taking into account the recommendations of Sergey Bushmanov, I'm providing an answer through this medium.
5
votes
1answer
653 views

Is there any difference between a weak learner and a weak classifier?

I am reading about Gradient Boosting, AdaBoost etc. I have found the following two concepts weak learner and weak classifier. Are they the same? If there is any difference what is it?
3
votes
1answer
48 views

Is there a real C4.5 implementation in Python ? (handling missing value)

To my understanding, C4.5 comes with 4 improvements compared to ID3: Handling missing values in both training data and "test" data, Handling continuous data Handling costs on attributes. The pruning ...
1
vote
0answers
31 views

Sklearn Decision Tree as weak learner in Adaboost not working properly

I'm trying to implement Adaboost algorithm with sklearn decision tree as the Weak Learner - at each step I want to choose one feature with one threshold to classify all samples. I have 1400 long ...
6
votes
5answers
637 views

Decision tree with final decision being a linear regression

Question: I want to implement a decision tree with each leaf being a linear regression, does such a model exist (preferable in sklearn)? Example case 1: Mockup data is generated using the formula: <...
1
vote
3answers
38 views

Will the MAE of testing data always be higher than MAE of training data?

On the Kaggle Course Page the chart below shows that MAE of testing data is always higher than MAE of training data. Why is this the case? Is it only limited to DecisionTreeRegressor model? Or the ...
3
votes
2answers
43 views

(Newbie) Decision Tree Classifier Splitting precedure

I have a dataset with 4 categorical features (Cholesterol, Systolic Blood pressure, diastolic blood pressure, and smoking rate). I use a decision tree classifier to find the probability of stroke. I ...
1
vote
1answer
21 views

Why there are no tutorials on calculating Gini impurity in a three-valued categorical feature variables?

Trying to learn here so please go easy on me if I asked dumb questions. As the title says, I was searching for a tutorial that calculates the Gini in a CART algorithm for a three-valued feature ...
4
votes
2answers
68 views

Why Decision Tree Classifier is not working with categorical value?

I am learning my way through this, so please be easy on me if you find any mistakes, I could really use a professional opinion here. Thx. I am trying to model a Decision Tree Classifier as part of an ...
3
votes
1answer
40 views

Decision Tree Classifier to classify values based on values of other columns

I have data with multiple labels, for example My X set is fromt second to third column, and I want to classify either first column or the last column, so I made my Y the last column. The goal is so ...
0
votes
1answer
69 views

How to select features which have non zero importance using SKLearn's Decision Trees Classifier?

I need to discard all the features which have zero importance and keep only those features which have non zero importance while implementing DecisionTreesClassifier. The feature importance here is ...
1
vote
1answer
21 views

DecisionTreeClassifier Integer Conditions, Integer Outcome Variable [closed]

Vague condition: "NumGoals >= 1.23" Preferred condition: "NumGoals > 1". Switched normalization off. Code: ...
1
vote
2answers
39 views

Can I force DecisionTreeClassifier to use integer conditions when the variable is integer?

I'm trying to visualize a decision tree in python for the purpose of explainability. I noticed that a condition like "NumGoals >= 1.23" could be quite vague for the user and I would much rather to see ...
1
vote
2answers
48 views

Use of decision trees for classifying images [closed]

I am new at Machine Learning and reading about it I wonder if it is possible (and convenient) to use decision trees to classify images. For instance, to classify faces
0
votes
0answers
18 views

Association, classification and decision rules - terminology?

Association rules as well as classification rules, both need to have a conjunction of values of the input attribute in their precondition. In the conclusion the association rules can have arbitrary ...
2
votes
1answer
47 views

Evaluate clustering by using decision tree unsupervised learning

I am trying to evaluate some clustering results that a company did for some data but they used an evaluation method for clustering that i have never seen before. So i would like to ask your opinion ...
0
votes
1answer
29 views

overfitting of regression tree from the beginning

I am trying to build a regression tree to model insurance claim frequencies. I have 36000 observations and 9 covariates. My model overfits right from the beginning!, ie as the cost complexity goes ...
5
votes
1answer
90 views

What is Pruning & Truncation in Decision Trees?

Pruning & Truncation As per my understanding Truncation: Stop the tree while it is still growing so that it may not end up with leaves containing very low data points. One way to do this is to ...
0
votes
1answer
24 views

What are the ways to identify a good attribute test while constructing a decision tree?

I'm working through a decision tree by hand to learn it. From my research, I have found the following three ways of determining which variables to split on: Minimum remaining values - The variable ...
1
vote
1answer
38 views

Why are my Decision Tree Leafs not pure?

I'm making a using DecisionTreeClassifier from SKlearn (v0.21.3) with its default settings, using Python. I do not want regularize it in any way, I want it to ...
0
votes
0answers
23 views

Should the LightGBM score match the regularization?

If I set the parameter objective to regression_l1 and set the metric to mean absolute error in ...
0
votes
1answer
54 views

Decision tree and random forest over fitting

I am working on a real state data set to predict the price of buying a house in Dubai based on area, no.of bedrooms, number of baths and the town which the house is in. All variables are numerical ...
3
votes
1answer
55 views

The output of Model from Decision Tree and Random Forests are different?

I have been made a model using both Decision Tree and Random Forest. But, when I tried to test the model on the same DataFrame the output is different. How is this possible? The data file from my ...

1 2 3 4 5 9