Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
1
vote
1answer
26 views

Explanation of random forest performance difference to when using categories and when using dummy variables

I have some hand coded feature which is a category with values "High", "Low", and "Normal". I created this feature myself and my problem performance (classification) ...
1
vote
1answer
27 views

How to select the split point for Continuous Attribute Age

For the above table, midpoints for possible split points are 22.5 and 35. I have calculated the entropy and gain for each value and 35 had the minimum Entropy and highest gain. Is it correct ? Given ...
0
votes
1answer
24 views

I am confused about these models

In the first problem, it's been told to accept the maximum number of good customers, if at least 98% of the customers that are do not repay their debt correctly identified. I am confused about what is ...
0
votes
0answers
4 views

Cost-complexity pruning: what the method does exactly?

The tree::prune.tree R function has a method parameter, described in the guide as: character string denoting the measure of ...
0
votes
1answer
16 views

Are there any tree-based models that use a genetic algorithm to generate the trees?

I have a large dataset (195 features x 20m samples) that I have trained using XGBoost. I would like to see if a genetic algorithm can beat XGBoost since the data has so much noise it is prone to ...
1
vote
1answer
9 views

Is there an advantage to use decision trees in OneVsAll than using “classical multiclass trees”?

I wonder if the is cases where it's better to use OneVsAll decision trees for multiclass classification ? I think that maybe it could be better for explainability of the model, but I didn't see ...
0
votes
1answer
29 views

Why my classification accuracy is high for both training and testing data?

I have a dataset with 10 features and 1 binary classification target. I tested this dataset with decision tree classifier. I did some basic check like missing values but the data looks clean. My ...
1
vote
1answer
23 views

KNN efficient implementation

The KNN algorithm is very handy and particularly suited to some of my problems, but I can't find any resources on how to implement it in production. As a comparative example, when I use a neural ...
0
votes
0answers
27 views

How to Determine Alpha values in Random Forest Classifier

I am working on a classification problem where I am applying various machine learning models. I have used DecisionTreeClassifier from Sklearn on my dataset using the following steps: Calculated alpha ...
0
votes
0answers
14 views

k-1 vs. k categories for categorical features in decision trees

I have a model with only categorical features. The outcome is binary. I am training binary classifiers. Most of them are coded already as 0/1 . Some have increased cardinality (ie 4 categories). For ...
0
votes
1answer
37 views

How does class_weight work in Decision Tree?

I am interested in Cost-Sensitive learning. And I am trying to understand how class_weight in DecisionTree works in terms of math. I read a lot of articles that ...
1
vote
1answer
35 views

Is feature importance from classification a good way to select features for clustering?

I have a large data set with many features (70). By doing preprocessing (removing features with too many missing values and those that are not correlated with the binary target variable) I have ...
1
vote
0answers
15 views

Looking for CART/ML model that works with relative data [duplicate]

I am a beginner at AI and ML. I have been given a dataset, where I have noticed the columns are relative to one another. So is there any CART or ML model that can work with relative data ? For example ...
0
votes
0answers
6 views

Tree Based Classification (XGBoost, LightGBM, etc) - Features from embeddings for sparse features?

I'm wondering if there is a possibility from using embeddings as inputs for tree based classification models? For example we have a field called type of food, and ...
1
vote
0answers
20 views

Decision trees vs Oblique decision trees

What are Oblique decision trees ? What are the differences between it and classic decision trees ?
1
vote
0answers
15 views

Information Gain of a Customer ID Attribute

Suppose I have a dataframe like the following ...
1
vote
0answers
19 views

Algorithm to determine a single output value based on multiple input values [closed]

The main challenge is the lack of data. Input values come from tests results of patients. A patient takes a breath test at an interval during a timespan. The result values can range from 0 to ~200, ...
0
votes
1answer
21 views

The notation of $splits(label)$ under Random Forest

On the "Fair Forests: Regularized Tree Induction to Minimize Model Bias", it is written that We propose a simple regularization approach to constructing a fair decision tree induction ...
1
vote
1answer
22 views

cost-complexity-pruning-path with pipeline

I'm using Kaggle's titanic set. I'm using pieplines and I'm trying to prune my decision tree and for that I want the cost_complexity_pruning_path. The last line of code produces the error: ...
0
votes
0answers
16 views

Why does Catboost outperform other boosting algorithms?

I have noticed while working with multiple datasets that catboost with its default parameters tends to outperform lightgbm or xgboost with its default parameters even on a tabular dataset with no ...
0
votes
2answers
79 views

what happens when a decision tree can't be split into further unit values?

Suppose I have a dataset A B C D 1 1 1 0 1 1 0 0 1 1 0 1 1 0 1 1 Here A,B,C,D are my independent features and D is my dependent feature. Now if I make a decision ...
3
votes
2answers
88 views

Gini impurity in decision tree (reasons to use it)

In a decision tree, Gini Impurity[1] is a metric to estimate how much a node contains different classes. It measures the probability of the tree to be wrong by sampling a class randomly using a ...
0
votes
0answers
13 views

Transforming binary data for decision trees

I have binary columns in my dataset (20) e.g. hot_weather, discount (y or no), where in each case 1 = yes no = 0. I am using this data on tree based methods. It is a regression problem and my RMSE is ...
0
votes
1answer
33 views

Which data science model is best for explainability for prediction problems?

Imagine you have to create a model to explain to stakeholders e.g. to predict price, weight, sales etc.. Which regression models offer the best in terms of explainability and interprability? ... Which ...
0
votes
0answers
31 views

Why does normalization improve my decision tree performances?

I have a regression problem for which I have to try several models, so I normalized my data and then tried to use a decision tree regressor (from sklearn.tree) and I noticed very good results (...
0
votes
0answers
21 views

Understanding min_samples_split and min_samples_leaf hyperparameters with DecisionTreeClassifier algorithm

My dataset consists of 775 samples and 117 features. The features represent developer skills (C++, Hadoop, AWS, etc.) and the output variable is a developer's profile (Frontend Developer, Quality ...
0
votes
1answer
20 views

How does tree-based algorithms handle linearly combined features?

While I am aware that tree-based algorithms (e.g., DT, RF, XGBoost) are 'immune' to multi-collinearity, how do they handle linearly combined features? For example, is there is any additional value or ...
3
votes
1answer
76 views

How is bayesian risk computed to prune decision trees?

I've been trying to follow this paper on Bayesian Risk Pruning. I'm not very familiar with this type of pruning, but I'm wondering a few things: (1) The paper describes risk-rates to be defined per ...
1
vote
1answer
43 views

Is there a closed formula/function for decision trees?

i've been studying gradient boosting so realize the pure algorithm requires a function F/model to get boosted.What is the explicit F on gradient boosting trees?
3
votes
0answers
43 views

How to properly do feature selection when comparing different models? [closed]

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
1
vote
1answer
43 views

How to model a supervised recommender system with varying data

Suppose there are 2000 movies and a company wants to recommend some movies (for example, at most 5 movies) to each visitor. The objective is to learn how to predict which movie will be selected if a ...
1
vote
0answers
24 views

Extracting rules in regression tree in Python

I don't know if this question is a complicated question or not. I wanna train a regression tree in which in leaves, linear regression is applied to predict. Then, when the tree and linear regression ...
0
votes
0answers
9 views

what is store in leaf node and split node in Annoy or any approximate nearest neighbor model built using tree?

Im trying to understand the working of annoy and have read the code _make_tree since im not from C++ background im trying hard to figure out the logic of whats stored in leaf node and split node ,you ...
0
votes
2answers
32 views

what happens to the sample points on pruned leaves

I think I understand what pruning is (in concept) in a decision tree. What is not so clear to me (in reading) is what happens to the pruned observations. What do you do with them? Are they just simply ...
1
vote
0answers
23 views

Calculating the lower and upper bounds forVC-dimension of a decision tree

I have a problem finding the lower and upper bounds of the decision tree. Suppose there is a decision tree with a hypothesis space of depth 2 and an input space with 10 variables (the variables take ...
1
vote
1answer
19 views

Strings/ features in Turicreate decision tree

I am trying to create a prediction model by using a decision tree with Turicreate. While my problem does involve numbers, it also involves strings and ultimately I want it to return the string 'true/...
2
votes
1answer
24 views

Optimum minimum number of instances in weka's j48 [closed]

There is a parameter named minnumobj in the options of the j48 tree algorithm in weka. This ...
0
votes
2answers
78 views

How to obtain the final values of a DecisionTreeRegressor in Scikit-Learn?

This page shows the paths in the decision trees in scikit-learn. After reaching the leaf nodes of the decision tree, where do we obtain the final resultant value?
0
votes
0answers
20 views

Is there a python package that includes decision tree structures that can be used with a genetic algorithm?

I'd like to use decision tree / forest but I need to use a special objective function that can't be differentiated and hence I can't use XGBOost etc. That leaves a genetic algorithm where I use the ...
3
votes
3answers
325 views

What is the best machine learning algorithm for large, noisy datasets with interaction between variables?

My initial thought was a neural network but I don't see how a neural network can properly predict interaction between variables (ie. x1 * x2) since each node is just a sum of previous inputs? Would a ...
0
votes
0answers
20 views

sklearn GradientBoostingClassifier provides unstable predictions

I am using sklearn.ensemble.GradientBoostingClassifier to build a rather simple model which predicts probabilities. I have simplified the problem to having only one numerical predictor and the outcome ...
1
vote
0answers
22 views

Manually building and visualising a decision tree

I am currently working on a decision tree for predicting the average of a binary outcome with a small number of categorical features (think predicting the survival rate in the titanic dataset). I am ...
0
votes
0answers
38 views

how to represent feature importance in xgboost in percentage?

I am looking for a way to represent the feature importance numbers in percentage. I read through articles and API documentation for XGboost in python and it gives me the feature importance score, ...
0
votes
1answer
21 views

how are split decisions for observations(not features) made in decision trees

I have read a lot of articles about decision trees, and every one of them only focused on telling how a feature/column is considered for split, based on criterion like gini index, entropy, chi-square ...
0
votes
1answer
21 views

How to build a classification pipeline that will pass to another model?

Not sure if the title explained it, but I am trying to build a pipeline where it's like a decision tree, but also not. Say for example, I had a picture. The model classified the picture, but now I ...
0
votes
1answer
44 views

Decision trees for anomaly detection

Problem From what I understand, a common method in anomaly detection consists in building a predictive model trained on non-anomalous training data, and perform anomaly detection using the error of ...
1
vote
1answer
79 views

Model retraining

I have trained my model with RandomForestRegressor, but now my training data is updated continuously. So I have to train my model with all the train data set i.e past and new data, or can I directly ...
0
votes
0answers
27 views

How to train decision trees for class imbalance binary classification?

sklearn documentations says Decision tree learners create biased trees if some classes dominate. It is therefore recommended to balance the dataset prior to fitting with the decision tree. Balance ...
1
vote
1answer
26 views

label encoding or one-hot encoding or none when using decision tree?

I've been learning about decision tree from multiple resources but still not fully understanding data preprocessing step. from https://www.youtube.com/watch?v=PHxYNGo8NcI&t=535s&ab_channel=...
1
vote
1answer
75 views

When using GridSearchCV with regression tree how to interpret mean_test_score?

I am using GridSearchCV to tune hyperparameters of regression decision tree. When I do, I get mean_test_score but I thought it would return mean MSE since it is a regressor. how to interpret ...

1
2 3 4 5
12