Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [decision-trees]

The tag has no usage guidance.

0
votes
0answers
5 views

create random forest using decision trees generated by different decision tree algorithms

We're planning to conduct a data mining study which will be using ID3, C4.5 and CART algorithms. Naturally, this would then generate three different decision trees. Would it be possible to create a ...
0
votes
0answers
6 views

how can decision rules be deduplicated?

I just fell on the interesting skope-rules package, which estimates decision rules from bagged trees. The best rules are then deduplicated, but the deduplication method is not detailed. Can someone ...
1
vote
1answer
7 views

AUC with sklearn vary each time script is started

I'm using the following code to perform a tree classification. I set up an int value for random_state in train_test_split ...
0
votes
0answers
9 views

ID3, C4.5 and CART random forest

I'm just curious, since the building blocks of random forests are decision trees, is it possible to build a random forest for each algorithm? (ex. a random forest using the ID3 algorithm)? I'm ...
1
vote
2answers
25 views

When creating a classification model, should predictors with little correlation to the response variable be included in the model?

I am building a predictive model designed to predict attrition within my organization. I am trying to decide whether to add certain predictors to my model. I used a Kruskal-Wallis rank sum test to ...
0
votes
1answer
14 views

How to migrate R decision tree to Java

I have trained a conditional inference decision tree in R using library party with function ctree and saved the model in an <...
0
votes
1answer
22 views

How I can interpret the attached tree?

Is the total sample size = 30.891 and the overall percentage of “yes” = 11,3%? How I can describe the leaves predicting "yes" outcome in term of explicative variables and values? What are the ...
1
vote
0answers
18 views

Do C 4.5 or C 5.0 perform multi way split or binary split?

Suppose that I am using a continuous variable as an independent variable (although a bad choice) for C 4.5 or C 5.0 (tree based classifier). I am having a hard time figuring out does C 4.5 or C 5.0 ...
0
votes
0answers
18 views

Better transformation technique to separate between classes? [closed]

I'm working on a classification problem. my two classes are highly overlapped. so, I'm looking for a new base ( a transformation ) that separates better between classes Which transformation ...
0
votes
1answer
17 views

Prediction interval for very small dataset

Does it make sense to calculate prediction interval for very small dataset (about 60 samples)? I know it's easy for linear regression. But it seems like for tree-based methods or non-linear methods it ...
0
votes
1answer
11 views

Décision tree, How to see under/over fitting with just looking at the leafs?

My question is: how with just looking at the leafs of a decision tree could you tell if the model is under/over-fitting? Any sort of advice will be helpful.
1
vote
1answer
32 views

Tuning svm and cart hyperparameters

I am trying to optimize the hyperparameters of SVM and CART with tune() function of e1071 R package, but I have a doubt. Should I tune the parameters on the training data, fit the model on the ...
1
vote
0answers
8 views

why does ID3 Decision tree algorithm not give the best decision tree?

I was going through ID3 algorithm, and what I believe is it incorporates Greedy Search rule to get come up with the decision tree. If it gives the best split possible at every stage, how does it not ...
1
vote
0answers
12 views

Merge one label with one information for classification problem or multi-label classification

I want to build a model to support decision making in order to propose or not loan insurance to clients. Because sometimes clients asking loan and loan insurance have less chance to have their loan ...
2
votes
0answers
18 views

“Binary Encoding” in “Decision Tree” / “Random Forest” Algorithms

Is it OK to use Binary Encoding in a dataset containing categorical columns with very high cardinalities? Some facts about my dataset: My dataset has ~170000 rows One of the categoric variables has ...
2
votes
0answers
21 views

Credit scoring using scorecardpy with XGBoost

I used XGBoost for scoring creditworthiness. At first I thought I could use predict_proba for scoring but then I saw that there was a module scorecardpy based on WOE to claculate code scoring. I tried ...
2
votes
1answer
20 views

Coefficient of determination is close to 1 but the value of RMSE is large. What does it mean?

I am working with the DecisionTreeRegressor and trying to understand how well the data fits the model. I calculated both RMSE ...
1
vote
0answers
15 views

Are there cases where tree based algorithms can do better than neural networks? [duplicate]

I trained an image auto-encoder on a large dataset, and now have for every image, an n-dimensional feature vector. This vector is not spatially correlated to the image. I now used this embedding space ...
1
vote
0answers
14 views

Decision tree to get difference in rates in two groups?

I have two sample groups of customers, each customer has 100s of features. For a single sample, i would use Decision Trees to find sub-groups that have a high churn rate. Thats easy. However, my ...
7
votes
1answer
60 views

Is max_depth in scikit the equivalent of pruning in decision trees?

I was analyzing the classifier created using a decision tree. There is a tuning parameter called max_depth in scikit's decision tree. Is this equivalent of pruning a decision tree? If not, how could I ...
1
vote
0answers
9 views

binary classification for counting - estimating the error on counts due to error on prediction score

Ok, so I have the following set up: I have a binary classification problem and I am classifying events into signal and background. Ultimately, I want to count how many background events and how many ...
4
votes
1answer
102 views

Why neural networks do not perform well on structured data?

I was recently working on some classification problem where decision trees performed better than neural networks. I had tried various combinations with neural networks altering the number of neurons / ...
0
votes
1answer
16 views

Does bagging create iid trees?

As the title suggests, I have a question regarding the trees produced through the bagging procedure. Namely, since the bootstrap samples created to fit trees on are independent and identically ...
2
votes
2answers
32 views

Why do we need a gain ratio

I'm learning about decision trees, and I feel like up till now I've understood them and the math behind them pretty well except for one thing: the gain ratio. As I understand, the gain ratio is ...
4
votes
1answer
69 views

When does decision tree perform better than the neural network?

I was experimenting with different modelling methods including KNN, Decision Trees, Neural Networks and SVN and trying to fit my data to see which works the best. To my surprise, the decision tree ...
1
vote
0answers
8 views

What happens if I do not encode the lables or classifiers in the data? [closed]

I have a data where the three variables are numerical and one variable is a string. I am using the simple decision tree algorithm. Read somewhere that the data in strings must be encoded with one hot ...
7
votes
1answer
50 views

How to decide the shape of input features, when each data file is of different length?

To help me understand the benefits and shortcomings of decision trees, KNN, Neural Networks, ...
1
vote
0answers
50 views

How decision trees work in Python

I am new to the field of machine learning. I have just recently learnt Decision Trees and started solving Titanic Survival problem from Kaggle Competition. I understood the algorithm behind decision ...
2
votes
2answers
20 views

Misclassification Rate for Random Forest Plateauing too Early

Using R, I have created 5 different random forest models using 5 different numbers of trees (3,10,30,100,300). My intention was to compute the misclassification rates of each of these models and plot ...
0
votes
1answer
65 views

Comparing Parameter Importance Across Models

I have a dataset with 20 features(columns that is). I create a few models pairs with a subset of these parameters. For example: If I have 6 columns (named A, B, C, D, E, F) with 10k lines of data, ...
2
votes
0answers
46 views

decision tree vs neural network for boolean function

Which structure is more powerful in terms of expressiveness (i.e. it can represent a given Boolean function, accurately) — a single-layer perceptron or a 2-layer decision tree? (There are 10 features)
2
votes
4answers
146 views

Are Decision Trees Robust to Outliers

I read that decision trees (I am using scikit-learn's classifier) are robust to outlier. Does that mean that I will not have any side-effect if I choose not to remove my outliers?
0
votes
1answer
34 views

Does running a Decision Tree classifier several times help?

To introduce, I am a novice in ML techniques. I recently had to write a scikit-learn based decision tree classifier to train on a real dataset. Someone suggested me ...
0
votes
1answer
33 views

Impact of sparse features on tree-based models

Say you have a highly imbalanced binary classification problem. Some of the features are binary features, where they're false most of the time, but when they're true they tend to be highly predictive (...
3
votes
1answer
72 views

Poker tournament winner prediction

I am trying to solve poker tournament winner prediction problem. I’ve millions of historical records in this format: Players ==> Winner P1,P2,P4,P8 ==> P2 P4,P7,P6 ==> P4 P6,P3,P2,P1 ==> P1 I ...
2
votes
3answers
47 views

Decision tree where identical set of features results in different outcomes

I am following the example described in this page to test my decision tree program. The initial data set is ...
1
vote
1answer
23 views

(Newbie) Decision Tree Randomness

I'm starting at Data Science and, to get something going, I just ran the code from Siraj Raval's Intro to Data Science video. He implements a simple Decision Tree Classifier but I couldn't help but ...
0
votes
0answers
86 views

how does XGBoost's exact greedy split finding algorithm determine candidate split values for different feature types?

Based on the paper by Chen & Guestrin (2016) "XGBoost: A Scalable Tree Boosting System", XGBoost's "exact split finding algorithm enumerates over all the possible splits on all the features to ...
3
votes
2answers
45 views

Ordinal Attributes in a Decision Tree

I'm reading the book Introduction to Data Mining by Tan, Steinbeck, and Kumar. In the chapter on Decision Trees, when talking about the "Methods for Expressing Attribute Test Conditions" the book says ...
1
vote
0answers
23 views

How to turn classification tree results into a GIS map

I'm new-ish to machine learning, so this could be a silly question. Apologies if so. The idea is is to predict groundwater occurrence based on a regression tree. This is my conceptual model: Target ...
0
votes
0answers
22 views

Soft Decision Tree implementation

Is there any library in Python or tool to generate Soft Decision Tree? I found sklearn package where I can easily create crisp Decision Tree using DecisionTreeClassifier() function. Do you know any ...
0
votes
1answer
54 views

Fraud detection using machine learning [closed]

I would like to implement a machine learning algorithm for data like this. ...
4
votes
6answers
431 views

I got 100% accuracy on my test set,is there something wrong?

I got 100% accuracy on my test set when trained using decision tree algorithm.but only got 85% accuracy on random forest Is there something wrong with my model or is decision tree best suited for the ...
1
vote
1answer
151 views

Gini Index in Regression Decision Tree

I want to implement my own version of the CART Decision Tree from scrach (to learn how it works) but I have some trouble with the Gini Index, used to express the purity of a dataset. More precisely, ...
1
vote
1answer
474 views

invalid character in identifier

I ran this code and this doesn't work, I'm using python 3 btw, I have checked the syntax a million times. I have installed all the necessary packages and all of them are up to date, here is the code I ...
1
vote
1answer
50 views

get_dump() leaf value and AUC

I have used Xgboost fitted a model with AUC around 0.73 and I printed out my last booster: ...
1
vote
0answers
32 views

Regression Decision Tree - Normalize or Split into Ranges a continuos feature

I have in my dataset a feature named distances which ranges goes from 200 to 12000 (more or less). Since the other features have got values under 50 I need to do ...
1
vote
0answers
303 views

Isolation Forest Feature Importance

As of scikit-learn version 0.19.1, there is no implementation for calculating feature importance in an Isolation Forest. I'm also having trouble finding any online resources proposing ways to get at ...
1
vote
0answers
60 views

How to interpret a trained Decision Tree [closed]

I built my first decision tree, to predict if students will pass or not - the data set - depending on 30 variables. Now I need to know how to read the decision tree, since many variables were strings ...
1
vote
1answer
30 views

Can I create random forest with RandomForestClassifier which will consist the same trees?

Based on answers to this question I should be able to build random forest with all the same trees by using bootstrap = False, max_features = None, random_state = 42 parameters. I wrote quick code to ...