Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [decision-trees]

The tag has no usage guidance.

0
votes
1answer
26 views

Random Forest, Duplicating Data increases Accuracy. Why?

I duplicated my training data for the random forest classifier (Sklearn) and the accuracy of the prediction increased by about 3%. Why?
0
votes
0answers
7 views

Proof of $R(t) \ge R(t_l) + R(t_r)$

In minimal complexity cost prunning of decision trees I found the following inequality $R(t) \ge R(t_l) + R(t_r)$ where $t$ is the current node and $t_l$, $t_r$ are the left node and the right node ...
0
votes
0answers
16 views

Generate Decision tree on one multi level categorical feature

I have 100K pairs of categorical observation(feature) and Label , for an example : As you can see, observations are over lapping the labels (different observations can have the same label) and ...
1
vote
0answers
7 views

How to reduce the number of rules in decision tree with Support and Confidence

The following is a set of rules in decison tree. How to reduce the number of rules in the set with Support and Confidence? If Ascites = 'Yes' then if Class = 'Live' then if Spiders = 'Yes' then if ...
0
votes
0answers
8 views

is max_depth except terminal node in decision tree?

I made the dicision tree with parameter max_depth = 5, max_leaf_node=15 but i get the model with 7 layer, which include root node, terminal node. I think I will get 6 layer because root_node + ...
0
votes
1answer
32 views

LightGBM - Why Exclusive Feature Bundling (EFB)?

I'm currently studying GBDT and started reading LightGBM's research paper: https://papers.nips.cc/paper/6907-lightgbm-a-highly-efficient-gradient-boosting-decision-tree.pdf In section 4. they explain ...
0
votes
1answer
18 views

Forcing a multi-label multi-class tree-based classifier to make more label predictions per document

I'm been experimenting with tree based classifiers for multi-label document classification. All the trees I've created, however, tend to predict only one or two labels per document. Whereas the ...
3
votes
1answer
41 views

How to prefer no choice instead of bad choice with sklearn decision tree

I'm using sklearn decision trees to classify documents in two possible types "type1" and "type2". I've isolated few features that seem pertinent and tried to combine them manually to evaluate the ...
1
vote
2answers
27 views

data pre-processing before image classification

I'm working on a machine learning project, Images classification (shape: 100 x 100)-> (vector of 10000), I did some pre-processing before applying decision trees algorithm , I got an accuracy of 55 % ...
4
votes
2answers
39 views

Why Decision trees performs better than logistic regression [closed]

I'm working on a machine learning project, a classification of (100 x 100) Images (every pixel contains 0 or 255), my training set contains 10000 examples (which I split into 2 parts 80% training/20% ...
3
votes
1answer
38 views

How max_features parameter works in DecisionTreeClassifier?

What is the parameter max_features in DecisionTreeClassifier responsible for? I thought it defines the number of features the ...
0
votes
1answer
34 views

Why xgboost can not deal with this simple sentence case?

There is only 1 feature dim. But the result is unreasonable. The code and data is below. The purpose of the code is to judge whether the two sentences are the same. In fact, the final input to the ...
1
vote
0answers
13 views

Decision Trees How to Calculate Entropy Gain for Continuous Values for split point?

I'm trying to calculate the entropy gain to decide the best decision split node, however I am having trouble understanding how to do this with doubles. I have columns listed out as A1 and A2 the ...
-1
votes
0answers
21 views

Linear Regression vs. Classification Trees

I had seen a lot of debate about use of linear regression and use of decision trees. There are no rules prescribing which one to use when, but could it be possible that methodologically both models ...
0
votes
0answers
6 views

Is there an equivalent library to “pydotplus” for Java?

I am looking for a way to visualize the resulting decision tree for a Java implementation of the Id3 algorithm. I stumbled upon this article which refers to how to visualize decision trees using ...
4
votes
3answers
49 views

Default parameters for decision trees give better results than parameters optimised using GridsearchCV

I am using Gridsearch for a DecisionTreeClassifier predicting a binary outcome. When I run fit and predict with default parameters, I get the following results: ...
2
votes
1answer
23 views

“help” decision tree by tying 2 features together

Assuming I have in my dataset 2 (or more) features that are for sure linked (for example: feature B indicates the amount of relevance of feature A), is there a way I could design a decision tree that ...
0
votes
2answers
34 views

Would a decision tree classifier be applicable here?

File from a situation where it is required to predict today’s stockprice from the stock prices of the previous three days: Could you use a decision tree classifier for this task? Why or why not?
1
vote
0answers
13 views

Is linear regression on the trees of XGBoost (rather than taking their mean) useful/popular?

Given training data $(\underline{x}_1, y_1),...,(\underline{x_N}, y_N)$, one can choose a variety of ensemble method for trees. These algorithms output a set of trees $T_1, ..., T_n$, and then the ...
1
vote
1answer
20 views

Decision Tree: Efficient splitting of nodes, minimize number of gini evaluations

I have a dataset specific problem where i need to use a splitting function other than gini_index. This requires me to re-write a decision tree from scratch. I have a working model, but itis highly ...
0
votes
0answers
8 views

pruned tree results in only terminal node

I am learning decision tree algorithm with binary classification. I have trained a decision tree and got pruned tree with just one feature as decision node. Now when I omit that particular feature ...
0
votes
0answers
21 views

“Histogram and binning” technique for categorical variables publication and implementations

H2O.ai have implemented the "histogram and binning" technique for efficient and accurate tree-building using categorical variables of high cardinality (>100): http://docs.h2o.ai/h2o/latest-stable/h2o-...
0
votes
0answers
11 views

create random forest using decision trees generated by different decision tree algorithms

We're planning to conduct a data mining study which will be using ID3, C4.5 and CART algorithms. Naturally, this would then generate three different decision trees. Would it be possible to create a ...
0
votes
0answers
10 views

how can decision rules be deduplicated?

I just fell on the interesting skope-rules package, which estimates decision rules from bagged trees. The best rules are then deduplicated, but the deduplication method is not detailed. Can someone ...
1
vote
1answer
14 views

AUC with sklearn vary each time script is started

I'm using the following code to perform a tree classification. I set up an int value for random_state in train_test_split ...
1
vote
2answers
32 views

When creating a classification model, should predictors with little correlation to the response variable be included in the model?

I am building a predictive model designed to predict attrition within my organization. I am trying to decide whether to add certain predictors to my model. I used a Kruskal-Wallis rank sum test to ...
0
votes
1answer
26 views

How to migrate R decision tree to Java

I have trained a conditional inference decision tree in R using library party with function ctree and saved the model in an <...
0
votes
1answer
26 views

How I can interpret the attached tree?

Is the total sample size = 30.891 and the overall percentage of “yes” = 11,3%? How I can describe the leaves predicting "yes" outcome in term of explicative variables and values? What are the ...
1
vote
0answers
21 views

Do C 4.5 or C 5.0 perform multi way split or binary split?

Suppose that I am using a continuous variable as an independent variable (although a bad choice) for C 4.5 or C 5.0 (tree based classifier). I am having a hard time figuring out does C 4.5 or C 5.0 ...
0
votes
1answer
24 views

Prediction interval for very small dataset

Does it make sense to calculate prediction interval for very small dataset (about 60 samples)? I know it's easy for linear regression. But it seems like for tree-based methods or non-linear methods it ...
0
votes
1answer
15 views

Décision tree, How to see under/over fitting with just looking at the leafs?

My question is: how with just looking at the leafs of a decision tree could you tell if the model is under/over-fitting? Any sort of advice will be helpful.
1
vote
1answer
46 views

Tuning svm and cart hyperparameters

I am trying to optimize the hyperparameters of SVM and CART with tune() function of e1071 R package, but I have a doubt. Should I tune the parameters on the training data, fit the model on the ...
1
vote
0answers
13 views

why does ID3 Decision tree algorithm not give the best decision tree?

I was going through ID3 algorithm, and what I believe is it incorporates Greedy Search rule to get come up with the decision tree. If it gives the best split possible at every stage, how does it not ...
1
vote
0answers
14 views

Merge one label with one information for classification problem or multi-label classification

I want to build a model to support decision making in order to propose or not loan insurance to clients. Because sometimes clients asking loan and loan insurance have less chance to have their loan ...
2
votes
0answers
25 views

“Binary Encoding” in “Decision Tree” / “Random Forest” Algorithms

Is it OK to use Binary Encoding in a dataset containing categorical columns with very high cardinalities? Some facts about my dataset: My dataset has ~170000 rows One of the categoric variables has ...
2
votes
0answers
114 views

Credit scoring using scorecardpy with XGBoost

I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
2
votes
1answer
26 views

Coefficient of determination is close to 1 but the value of RMSE is large. What does it mean?

I am working with the DecisionTreeRegressor and trying to understand how well the data fits the model. I calculated both RMSE ...
1
vote
0answers
17 views

Are there cases where tree based algorithms can do better than neural networks? [duplicate]

I trained an image auto-encoder on a large dataset, and now have for every image, an n-dimensional feature vector. This vector is not spatially correlated to the image. I now used this embedding space ...
1
vote
0answers
17 views

Decision tree to get difference in rates in two groups?

I have two sample groups of customers, each customer has 100s of features. For a single sample, i would use Decision Trees to find sub-groups that have a high churn rate. Thats easy. However, my ...
8
votes
2answers
109 views

Is max_depth in scikit the equivalent of pruning in decision trees?

I was analyzing the classifier created using a decision tree. There is a tuning parameter called max_depth in scikit's decision tree. Is this equivalent of pruning a decision tree? If not, how could I ...
1
vote
0answers
11 views

binary classification for counting - estimating the error on counts due to error on prediction score

Ok, so I have the following set up: I have a binary classification problem and I am classifying events into signal and background. Ultimately, I want to count how many background events and how many ...
5
votes
1answer
177 views

Why neural networks do not perform well on structured data?

I was recently working on some classification problem where decision trees performed better than neural networks. I had tried various combinations with neural networks altering the number of neurons / ...
0
votes
1answer
20 views

Does bagging create iid trees?

As the title suggests, I have a question regarding the trees produced through the bagging procedure. Namely, since the bootstrap samples created to fit trees on are independent and identically ...
2
votes
2answers
59 views

Why do we need a gain ratio

I'm learning about decision trees, and I feel like up till now I've understood them and the math behind them pretty well except for one thing: the gain ratio. As I understand, the gain ratio is ...
4
votes
1answer
136 views

When does decision tree perform better than the neural network?

I was experimenting with different modelling methods including KNN, Decision Trees, Neural Networks and SVN and trying to fit my data to see which works the best. To my surprise, the decision tree ...
1
vote
0answers
9 views

What happens if I do not encode the lables or classifiers in the data? [closed]

I have a data where the three variables are numerical and one variable is a string. I am using the simple decision tree algorithm. Read somewhere that the data in strings must be encoded with one hot ...
7
votes
1answer
52 views

How to decide the shape of input features, when each data file is of different length?

To help me understand the benefits and shortcomings of decision trees, KNN, Neural Networks, ...
1
vote
0answers
102 views

How decision trees work in Python

I am new to the field of machine learning. I have just recently learnt Decision Trees and started solving Titanic Survival problem from Kaggle Competition. I understood the algorithm behind decision ...
2
votes
2answers
23 views

Misclassification Rate for Random Forest Plateauing too Early

Using R, I have created 5 different random forest models using 5 different numbers of trees (3,10,30,100,300). My intention was to compute the misclassification rates of each of these models and plot ...