Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
0
votes
1answer
10 views

How to explain MAE/MSE at each node of decision tree for regression in sklearn python?

If the mean value at any node is 60 and MSE = 169 so RMSE is 13. Can I conclude that the error at my node is 60 +-13 i.e my values in this particular sample split ranges from 60-13 to 60+13. If not , ...
0
votes
0answers
12 views

Multiclass ROC Curve using DecisionTreeClassifier

I built a DecisionTreeClassifier with custom parameters to try to understand what happens modifying them and how the final model classifies the instances of the iris dataset. Now My task is to create ...
0
votes
1answer
18 views

Same confusion matrix when changing DecisionTreeClassifier parameters

I'm trying to build my first Decision Tree Classifier using the Iris dataset in the sklearn library. This is my first sample code: ...
1
vote
1answer
18 views

Random selection of variables in each run of python sklearn decision tree (regressio )

When I put random_state = None and run Decision tree for regression in python sklearn, it takes different variables to build tree each time? Shouldn't there be only ...
2
votes
2answers
25 views

Why is large decision tree likely to overfit

My lecture slide told me that if we don't prune the regression tree, then the tree likely to over-fit. So, I wonder why would that happen? Is that because if the tree grows too large, we would end up ...
4
votes
2answers
31 views

Sklearn: applying cost complexity pruning along with pipeline

I have a data set with categorical variables. I have defined a decision tree algorithm and transformed these columns to numerical equivalent using one hot encoding functionality in sklearn: Create ...
1
vote
1answer
18 views

Tree-based algorithms and ordinal features

For tree-based methods (e.g., DT, Random Forest, Gradient boosting, etc.), does the conversion interval of an ordinal feature to continuous matter matters? (I can see why it matters for linear model, ...
-1
votes
0answers
21 views

Confusion with the solution for this decision theory problem [closed]

$\textbf{I am given the following Decision Theory question:}$ Given a loss matrix with elements $L_{kj}$, the expected risk is minimized if, for each $x$, we choose the class that minimizes $\sum_{k} ...
0
votes
0answers
18 views

How much does each tree of gradient boosting contribute to the global feature importance?

Let's say we are training a GBDT in the Titanic dataset. We have 3 trees in the GBDT. You extract the first tree and calculate the feature importance (no matter if cover, gain...), and Age importance =...
1
vote
1answer
20 views

Counting the number of trainable parameters in a gradient boosted tree

I recently ran the gradient boosted tree regressor using scikit-learn via: GradientBoostingRegressor() This model depends on the following hyperparameters: Estimators ($N_1$) Min Samples Leaf ($N_2$...
1
vote
1answer
27 views

How to find the feature regions where each label is the most expected when using decision trees?

Given a decision tree for classification for example this one: What is the way to find the feature domain (petal and sepal width and length) where a sample would most likely occur in the feature ...
0
votes
1answer
26 views

Machine learning prediction algorithms for new baby names in different religions [closed]

https://www.google.com/search?sxsrf=ALeKk005ib1xM_GLwXjFCRszcrdLdlyfPg%3A1602062576170&ei=8Ih9X4XuCYyZ4-EPn5u7sAs&q=new+born+baby+names+prediction&oq=new+born+baby+names+prediction&...
1
vote
2answers
28 views

Is fitting two RandomForestClassifiers 500 trees each and average their predicted probabilities on the test set more performant than one with 1000?

If I fit two RandomForestClassifiers 500 trees each and average their predicted probabilities on the test set, would it have better results than fitting a RandomForestClassifier with 1000 trees and ...
2
votes
3answers
712 views

Using a random forest, would a RandomForest performance be less if I drop the first or the last tree?

Suppose I've trained a RandomForest model with 100 trees. I then have two cases: I drop the first tree in the model. I drop the last tree in the model. Would the model performance be less in the ...
5
votes
2answers
64 views

How to use “tree boosting” with a data-driven loss function

We have a problem which has a data-driven (non-analytical) loss function. Our target contains whole numbers between 0 and 20 (the target is inherently discrete), although larger values are possible, ...
1
vote
2answers
30 views

Can machine learning models treat a vector as a whole feature to learn

We know a ML model naturally takes a feature vector with real valued elements as input and learn to predict. But can it treat a fixed-size vector as a whole feature to learn? For example, when using a ...
0
votes
1answer
17 views

Using Gini index, how to calculate the probability of correctly classifying a new randomly selected case to the highest probability class? [closed]

I have the following binary Decision Tree: Can you please explain how can I report this tree to a person who only understands probabilities? If ca=1 and cp_4.0=1, what’s the probability of Yes HD?
2
votes
1answer
25 views

Can I do bagging method as improvement technique to decision tree in research?

Bagging use decision tree as base classifier. I want to use bagging with decision tree(c4.5) as base as the method that improve decision tree(c4.5) in my research that solve problem overfitting. Is ...
1
vote
1answer
18 views

Decision Tree Regressor: domain of the y variable

just wondering about a thing. suppose you fit a Decision Tree Regressor and your training y variable has got a domain that spans ...
1
vote
1answer
39 views

Interpreting decision tree results after target encoding

I am not sure how to interpret the results of my decision tree after I had used target encoding, could someone clarify? The example below doesn't need target encoding just for explanation of my ...
0
votes
0answers
13 views

Collapse categorical variable to reduce levels using a decision tree

I am using zip codes as an independent variable as part of a binary classification problem. Naturally, this feature has many different levels (around 2,000), so I was wondering if there is a ...
0
votes
1answer
31 views

NN Making Poor Averaging Fit whilst LGBM Regressor Fits Perfectly

I have a simple toy dataset for which the features have been encoded using a Encoder-Decoder NN. I am using the hidden feature vector from the Encoder as the X input for training a 1-step lookahead ...
-1
votes
1answer
29 views

How to identify Overfitting in RandomForestClassifier?

Im building a sentiment classification model using RandomForestClassifier. I got the training accuracy of 99.65 & cross-validation( RepeatedStratifiedKFold-5 folds) accuracy of 97.29. I used f1 ...
3
votes
1answer
34 views

XGBoost Tree 'starting feature break'

I am fairly new to learning the XGBoost algorithm and had a question about how the algorithm knows which feature to break the tree on first. Here is my understanding (and please correct me if I'm ...
0
votes
0answers
30 views

Classification and decision trees for beginners

I'm working with a social science dataset about the academic performance of university students. There are many variables on the dataset (Gender, Vital records, education level of parents, attendance ...
1
vote
0answers
30 views

Prediction in CART Decision Trees

I was studying the algorithm of CART (classification and regression trees), but the formula of the prediction is irritating me. First we have the following definition: Let $X:={x_1,...,x_N} \subset \...
2
votes
1answer
41 views

How to handle a regression problem with skewed target and only few high values?

I'm currently tackling a regression problem with skewed target variable (presented below). Naturally, my first idea was to transform the target with natural logarithm as it'll probably help both ...
1
vote
1answer
55 views

splitting mechanism with one hot encoded variables (tree based/boosting)

I am using xgboost and have a categorical unordered feature with 25 levels. So when i apply one hot encoding i have 25 columns. This introduces alot of sparsity. Even more unusual, my feature ...
4
votes
1answer
89 views

How do we decide between XGBoost, RandomForest and Decision tree?

What do we take into consideration while deciding which technique should be used when dealing with a particular dataset? I understand that there isn't any hard and fast rule to this. Do we use XGBoost ...
1
vote
1answer
47 views

Why Adaboost SAMME needs f to be estimable?

I am trying to understand the mathematics behind SAMME AdaBoost: At some stage, the paper adds a constraint for f to be estimable: I do not understand why this is ...
3
votes
2answers
110 views

Is Label Encoding with arbitrary numbers ever useful at all?

From what I read online, there seems to be some confusion regarding the taxonomy and the terms used, so to avoid misunderstanding I'm going to define them here: Label Encoding - encoding a nominal ...
2
votes
0answers
17 views

Output value of a gradient boosting decision tree node that has just a single example in it

The general gradient boosting algorithm for tree-based classifiers is as follows: Input: training set ${\displaystyle \{(x_{i},y_{i})\}_{i=1}^{n},}{\displaystyle \{(x_{i},y_{i})\}_{i=1}^{n},}$a ...
1
vote
1answer
118 views

decision -tree regression to avoid multicollinearity for regression model?

I read in comments a recommendation for decision tree´s instead of linear models like neural network, when the dataset has many correlated features. Because to avoid multicollinearity. A similar ...
4
votes
1answer
164 views

feature scaling xgbRegressor

I read for example in this answer: Does the performance of GBM methods profit from feature scaling? that scaling doesn´t affect the performance of any tree-based method, not for lightgbm,xgboost,...
1
vote
1answer
17 views
3
votes
1answer
35 views
1
vote
0answers
18 views

Decision Boundary with random new observations vs observations from test set

I'm trying to plot decision boundary for Decision Tree classifier. Classifier is trained on training set, and decision boundary (contour) using random new observations and observations from test set ...
1
vote
1answer
70 views

What can weka do that python and sklearn can't?

I would like to build a variety of classification and regression decision trees. My use case focuses on extraction and communication of decision rules. Previously weka was used in my organisation for ...
3
votes
2answers
153 views

Decision Tree : how to determine target in a model with no labels?

I am studying classification algorithms using decision tree approach in Python. I would have some questions on this topic, specifically regarding the target (y) in my dataset. I have a dateset made by ...
3
votes
1answer
91 views

Decision tree and SVM for text classification - theory

I used 4 classifiers for my text data: NB, kNN, DT and SVM. As for NB and kNN I fully understand how they work with text - how we can count probabilities for all words in NB and how to use similarity ...
0
votes
0answers
13 views

Time Series Predictive Model

I have a dataset similar to the following one: ...
4
votes
1answer
142 views

What happens if at leaf node both classes have same number of samples?

I analyzed a small dataset which had three features, so I kept max_depth of decision tree to be 3, in doing so I found it something intresting, there was a leaf node which had number of samples of ...
1
vote
1answer
14 views

How do I deal with data that has only limited target values?

I'm currently working on a small project using the D1NAMO dataset (1). I want to predict the glucose level (that is given in the dataset) based on several features: accelerometer data, heartbeat (ECG) ...
1
vote
1answer
20 views

How Calculate Effect (percentage) label of the input variables on the output variable by DecisionTreeClassifier

a description problem below. I have 10 words like X1 , X2 , X3 , ... , X10 and three Label like short , long , hold. My problem is that how calculate Effect (percentage) label of the input variables ...
0
votes
1answer
29 views

Zero-inflated independent feature in tree-based models

What is the best approach to include a zero-inflated continuous independent feature (e.g., 90% of the values are Zero, 10% are >0) in a Tree-based models (DT, random forest, gradient boosting. etc)....
1
vote
1answer
30 views

How does bagging help reduce the variance

I learned that bagging helps reduce variance by averaging but I couldn't understand this. Can someone explain this intuitively?
0
votes
0answers
13 views

Building Uplift Tree using boosting

I want to build an Uplift Model for multiple treatments. To get a good model, I would like to use boosting. How is it possible to use boosting with uplift modeling although we can't really calculate ...
3
votes
3answers
79 views

Better approach to assign values to determine potential fake sentences

I am trying to assign different values for each sentences based on information about the presence of hashtags, upper case letters/words (e.g. HATE) and some others. I created a data frame which ...
1
vote
1answer
25 views

Interpreting high precision and very low recall score

I was training model on a very imbalanced dataset with 80:20 ratio of two classes. The dataset has thousands of rows and I trained the model using ...
1
vote
0answers
14 views

Which models implicitly consider interaction between features?

I would like to understand more how different models (NN and RF specifically, but any other as well) consider interaction between features in tabular data? For example, can the model figure out while ...

1
2 3 4 5
11