Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
1
vote
1answer
43 views

Is feature importance from classification a good way to select features for clustering?

I have a large data set with many features (70). By doing preprocessing (removing features with too many missing values and those that are not correlated with the binary target variable) I have ...
1
vote
0answers
15 views

Looking for CART/ML model that works with relative data [duplicate]

I am a beginner at AI and ML. I have been given a dataset, where I have noticed the columns are relative to one another. So is there any CART or ML model that can work with relative data ? For example ...
0
votes
0answers
48 views

Tree Based Classification (XGBoost, LightGBM, etc) - Features from embeddings for sparse features?

I'm wondering if there is a possibility from using embeddings as inputs for tree based classification models? For example we have a field called type of food, and ...
1
vote
0answers
22 views

Decision trees vs Oblique decision trees

What are Oblique decision trees ? What are the differences between it and classic decision trees ?
1
vote
0answers
33 views

Information Gain of a Customer ID Attribute

Suppose I have a dataframe like the following ...
1
vote
0answers
22 views

Algorithm to determine a single output value based on multiple input values [closed]

The main challenge is the lack of data. Input values come from tests results of patients. A patient takes a breath test at an interval during a timespan. The result values can range from 0 to ~200, ...
0
votes
1answer
23 views

The notation of $splits(label)$ under Random Forest

On the "Fair Forests: Regularized Tree Induction to Minimize Model Bias", it is written that We propose a simple regularization approach to constructing a fair decision tree induction ...
1
vote
1answer
70 views

cost-complexity-pruning-path with pipeline

I'm using Kaggle's titanic set. I'm using pieplines and I'm trying to prune my decision tree and for that I want the cost_complexity_pruning_path. The last line of code produces the error: ...
0
votes
1answer
45 views

Why does Catboost outperform other boosting algorithms?

I have noticed while working with multiple datasets that catboost with its default parameters tends to outperform lightgbm or xgboost with its default parameters even on a tabular dataset with no ...
0
votes
2answers
152 views

what happens when a decision tree can't be split into further unit values?

Suppose I have a dataset A B C D 1 1 1 0 1 1 0 0 1 1 0 1 1 0 1 1 Here A,B,C,D are my independent features and D is my dependent feature. Now if I make a decision ...
3
votes
2answers
161 views

Gini impurity in decision tree (reasons to use it)

In a decision tree, Gini Impurity[1] is a metric to estimate how much a node contains different classes. It measures the probability of the tree to be wrong by sampling a class randomly using a ...
0
votes
0answers
18 views

Transforming binary data for decision trees

I have binary columns in my dataset (20) e.g. hot_weather, discount (y or no), where in each case 1 = yes no = 0. I am using this data on tree based methods. It is a regression problem and my RMSE is ...
0
votes
1answer
39 views

Which data science model is best for explainability for prediction problems?

Imagine you have to create a model to explain to stakeholders e.g. to predict price, weight, sales etc.. Which regression models offer the best in terms of explainability and interprability? ... Which ...
0
votes
0answers
43 views

Why does normalization improve my decision tree performances?

I have a regression problem for which I have to try several models, so I normalized my data and then tried to use a decision tree regressor (from sklearn.tree) and I noticed very good results (...
0
votes
0answers
43 views

Understanding min_samples_split and min_samples_leaf hyperparameters with DecisionTreeClassifier algorithm

My dataset consists of 775 samples and 117 features. The features represent developer skills (C++, Hadoop, AWS, etc.) and the output variable is a developer's profile (Frontend Developer, Quality ...
0
votes
1answer
24 views

How does tree-based algorithms handle linearly combined features?

While I am aware that tree-based algorithms (e.g., DT, RF, XGBoost) are 'immune' to multi-collinearity, how do they handle linearly combined features? For example, is there is any additional value or ...
4
votes
1answer
89 views

How is bayesian risk computed to prune decision trees?

I've been trying to follow this paper on Bayesian Risk Pruning. I'm not very familiar with this type of pruning, but I'm wondering a few things: (1) The paper describes risk-rates to be defined per ...
1
vote
1answer
56 views

Is there a closed formula/function for decision trees?

i've been studying gradient boosting so realize the pure algorithm requires a function F/model to get boosted.What is the explicit F on gradient boosting trees?
3
votes
0answers
47 views

How to properly do feature selection when comparing different models? [closed]

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
2
votes
1answer
48 views

How to model a supervised recommender system with varying data

Suppose there are 2000 movies and a company wants to recommend some movies (for example, at most 5 movies) to each visitor. The objective is to learn how to predict which movie will be selected if a ...
1
vote
0answers
29 views

Extracting rules in regression tree in Python

I don't know if this question is a complicated question or not. I wanna train a regression tree in which in leaves, linear regression is applied to predict. Then, when the tree and linear regression ...
0
votes
0answers
13 views

what is store in leaf node and split node in Annoy or any approximate nearest neighbor model built using tree?

Im trying to understand the working of annoy and have read the code _make_tree since im not from C++ background im trying hard to figure out the logic of whats stored in leaf node and split node ,you ...
0
votes
2answers
33 views

what happens to the sample points on pruned leaves

I think I understand what pruning is (in concept) in a decision tree. What is not so clear to me (in reading) is what happens to the pruned observations. What do you do with them? Are they just simply ...
1
vote
0answers
26 views

Calculating the lower and upper bounds forVC-dimension of a decision tree

I have a problem finding the lower and upper bounds of the decision tree. Suppose there is a decision tree with a hypothesis space of depth 2 and an input space with 10 variables (the variables take ...
1
vote
1answer
28 views

Strings/ features in Turicreate decision tree

I am trying to create a prediction model by using a decision tree with Turicreate. While my problem does involve numbers, it also involves strings and ultimately I want it to return the string 'true/...
2
votes
1answer
49 views

Optimum minimum number of instances in weka's j48 [closed]

There is a parameter named minnumobj in the options of the j48 tree algorithm in weka. This ...
0
votes
2answers
301 views

How to obtain the final values of a DecisionTreeRegressor in Scikit-Learn?

This page shows the paths in the decision trees in scikit-learn. After reaching the leaf nodes of the decision tree, where do we obtain the final resultant value?
0
votes
0answers
21 views

Is there a python package that includes decision tree structures that can be used with a genetic algorithm?

I'd like to use decision tree / forest but I need to use a special objective function that can't be differentiated and hence I can't use XGBOost etc. That leaves a genetic algorithm where I use the ...
3
votes
3answers
578 views

What is the best machine learning algorithm for large, noisy datasets with interaction between variables?

My initial thought was a neural network but I don't see how a neural network can properly predict interaction between variables (ie. x1 * x2) since each node is just a sum of previous inputs? Would a ...
0
votes
0answers
21 views

sklearn GradientBoostingClassifier provides unstable predictions

I am using sklearn.ensemble.GradientBoostingClassifier to build a rather simple model which predicts probabilities. I have simplified the problem to having only one numerical predictor and the outcome ...
1
vote
0answers
25 views

Manually building and visualising a decision tree

I am currently working on a decision tree for predicting the average of a binary outcome with a small number of categorical features (think predicting the survival rate in the titanic dataset). I am ...
0
votes
0answers
70 views

how to represent feature importance in xgboost in percentage?

I am looking for a way to represent the feature importance numbers in percentage. I read through articles and API documentation for XGboost in python and it gives me the feature importance score, ...
0
votes
1answer
53 views

how are split decisions for observations(not features) made in decision trees

I have read a lot of articles about decision trees, and every one of them only focused on telling how a feature/column is considered for split, based on criterion like gini index, entropy, chi-square ...
0
votes
1answer
22 views

How to build a classification pipeline that will pass to another model?

Not sure if the title explained it, but I am trying to build a pipeline where it's like a decision tree, but also not. Say for example, I had a picture. The model classified the picture, but now I ...
0
votes
1answer
72 views

Decision trees for anomaly detection

Problem From what I understand, a common method in anomaly detection consists in building a predictive model trained on non-anomalous training data, and perform anomaly detection using the error of ...
1
vote
1answer
164 views

Model retraining

I trained my model with RandomForestRegressor(), but now my training data is updated continuously. So I have to train my model with all the train data set i.e past ...
1
vote
1answer
99 views

label encoding or one-hot encoding or none when using decision tree?

I've been learning about decision tree from multiple resources but still not fully understanding data preprocessing step. from https://www.youtube.com/watch?v=PHxYNGo8NcI&t=535s&ab_channel=...
1
vote
1answer
327 views

When using GridSearchCV with regression tree how to interpret mean_test_score?

I am using GridSearchCV to tune hyperparameters of regression decision tree. When I do, I get mean_test_score but I thought it would return mean MSE since it is a regressor. how to interpret ...
-1
votes
1answer
45 views

Interpreting MSE in regression Trees

I am using regression tree to predict target variable(continuous). I've use one-hot encoding for all categorical features and applied standard scaler to all numerical features. After all that I train ...
1
vote
1answer
154 views

Fitting probabilities in scikit-learn RandomForestClassifier

This is a newbie questions, so please bear with me. Given this random forest model: ...
2
votes
2answers
226 views

How to decode encoded labels in Decision tree classifier

I have some dataset with procurements of organization where actually i'm working. The aim is to find most important features that describe why some processes of purchases is succesful, and why not ...
0
votes
1answer
32 views

how do tree based methods deal with missing feature columns?

all, i have trained a model using xgboost. Some of the features are one hot encoded e.g. currency where it is either gbp or usd. it seems that when i output the feature importance gbp and usd were in ...
1
vote
0answers
23 views

How many positive responses are good enough for building a marketing response model when the response rate is low(0.5%)

We are planning a marketing campaign to collect data and the response rates for a random sample. Total population size is 10 million and historically, response rates are low (0.5 - 0.65 %). How long ...
0
votes
1answer
43 views

Correctness of a ROC Curve

I've built a Decision Tree Classifier to practice with the sklearn library. My first task was to shuffle the iris dataset and split it keeping only the last 10 elements for the test. Then, after the ...
1
vote
1answer
416 views

How to explain MAE/MSE at each node of decision tree for regression in sklearn python? [closed]

If the mean value at any node is 60 and MSE = 169 so RMSE is 13. Can I conclude that the error at my node is 60 +-13 i.e my values in this particular sample split ranges from 60-13 to 60+13. If not , ...
1
vote
1answer
308 views

Multiclass ROC Curve using DecisionTreeClassifier

I built a DecisionTreeClassifier with custom parameters to try to understand what happens modifying them and how the final model classifies the instances of the iris dataset. Now My task is to create ...
0
votes
1answer
67 views

Same confusion matrix when changing DecisionTreeClassifier parameters

I'm trying to build my first Decision Tree Classifier using the Iris dataset in the sklearn library. This is my first sample code: ...
1
vote
1answer
26 views

Random selection of variables in each run of python sklearn decision tree (regressio )

When I put random_state = None and run Decision tree for regression in python sklearn, it takes different variables to build tree each time? Shouldn't there be only ...
2
votes
2answers
123 views

Why is large decision tree likely to overfit

My lecture slide told me that if we don't prune the regression tree, then the tree likely to over-fit. So, I wonder why would that happen? Is that because if the tree grows too large, we would end up ...
5
votes
2answers
270 views

Sklearn: applying cost complexity pruning along with pipeline

I have a data set with categorical variables. I have defined a decision tree algorithm and transformed these columns to numerical equivalent using one hot encoding functionality in sklearn: Create ...

1 2
3
4 5
14