Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
0
votes
1answer
24 views

What is the meaning of the Gini Index?

I'm studying random forest models, but I don't understand what the Gini index is and what it's for. Does anyone have any material on this or can give me an explanation? Thanks!
0
votes
1answer
11 views

Additional business rules in ensemble methods (RF, Boosted Trees)

How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees? These additional business rules can be ...
1
vote
0answers
13 views

How are regression trees fit in gradient boosting for classification?

What I understood is that even gradient boosting for binary classification we use regression trees. The first value we calculate is constant = log(odds). For the rest of the trees we try to fit ...
1
vote
1answer
33 views

Scikit-learn's implementation of AdaBoost

I am trying to implement the AdaBoost algorithm in pure Python (or using NumPy if necessary)....
0
votes
1answer
18 views

Does hyperparameter tuning of Decision Tree then use it in Adaboost individually vs Simultaneously yield the same results?

So, my predicament here is as follows, I performed hyperparameter tuning on a standalone Decision Tree classifier, and I got the best results, now comes the turn of Standalone Adaboost, but here is ...
2
votes
2answers
26 views

confused on "real score" vs "decision value" in classification trees

I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is: A ...
1
vote
0answers
11 views

Classification for Ordinal labels - what tree-based methds can i use?

I have a label that has a natural ordering e.g. 0,1,2,3 where 0 is the worst activity measure and 3 is the best. For each label given by the model i need to also give the probability that it belongs ...
3
votes
2answers
48 views

Is it possible to 'group features' for a decision tree model?

At each node of a decision tree, we must choose a collection of features to split along. Suppose we know a priori that the features can be partitioned into subsets that are 'correlated', i.e. this ...
0
votes
0answers
20 views

Is it always that lower tree with higher bias but higher tree with higher varaince

When dealing with bias and variance trade-offs, I always hear that in tree models: shallow tree = high bias but low variance, deep tree = low bias but high variance. Someone may also quote from high ...
3
votes
1answer
55 views

Multi-target regression tree with additional constraint

I have a regression problem where I need to predict three dependent variables ($y$) based on a set of independent variables ($x$): $$ (y_1,y_2,y_3) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \...
0
votes
0answers
14 views

User Churn Rate analysis - Binary classification

I have a dataset which has the logs of user clicks. This is a trail version(2 months) of the software. Users can use a special feature during this trail period to improve their sales. The number of ...
0
votes
0answers
13 views

Is there any function of targets in binary classification decision tree

I was recently learning about decision tree and stumbled across a question which might be very silly but i am unable to understand it . That is if for a binary classification problem splits are used ...
0
votes
1answer
26 views

Orthogonal Decision Boundary in Decision Tree

I was reading the limitations of decision trees. One of them was that, for classification problems, decision trees produce only orthogonal decision boundaries. Could anyone please explain what an ...
2
votes
0answers
22 views

Building machine learning models whilst penalizing them for complexity

I come from a predictive modelling background, where it's common to use differential equations to model physical or chemical or biological processes. Commonly to avoid overfitting people use AIC and ...
0
votes
2answers
23 views

How to create classification decision trees on a dataset that has both numerical and categorical variables?

I am quite new to Data Science and learning things hands-on in the job. I am a fraud analyst and my job is to predict whether an application is fraudulent or not based on data. Before moving on to ...
0
votes
0answers
24 views

LightGBM boosting and bagging parameters

When training a gradient boosted decision tree model, I can use the LightGBM package to efficiently train my model. It's possible to define the hyperparameter search space with eg. ...
0
votes
0answers
31 views

Regression trees for extrapolating time series data

This is a regression problem that involves predicting the price of e.g. aluminum, oil, strawberries. I have hourly and half hourly data for the weather and up to 10 different socioeconomic variables (...
1
vote
1answer
22 views

Getting both results and probabilities running scikit learn random forest

I have a scikit learn RandomForestClassifier that returns 0s and 1s: ...
0
votes
0answers
35 views

Decision Tree taking too long to execute

I am training a Decision Tree Regressor on a relatively small data. The dimensions of my train and test sets are (34164, 10) and (8514, 10). Here is the relevant code: ...
0
votes
1answer
136 views

Combining heterogeneous numerical and text features

We want to solve a regression problem of the form "given two objects $x$ and $y$, predict their score (think about it as a similarity) $w(x,y)$". We have 2 types of features: For each ...
1
vote
1answer
41 views

Decision Trees and Categorical Feature Labelling

I am working on a decision tree model and trying to decide how best to handle categorical features. The features in my dataset are generally high in cardinality and I have found that ordinal labeling ...
0
votes
0answers
15 views

How to get variance for regression tree fit?

Suppose the true function is a tree such that: $$f(x)=\sum_{j=1}^{J}b_j I(x \in R_j)+e_i$$ where $b_j=E(y|x \in R_j)$ ,$E(e_i)=0$ and $R_j$ as terminal node. Suppose we got a fit for this tree via ...
0
votes
1answer
89 views

Is feature importance in XGBoost or in any other tree based method reliable?

This question is quite long, if you know how feature importance to tree based methods works i suggest you to skip to text below the image. Feature importance (FI) in tree based methods is given by ...
0
votes
1answer
31 views

Standardizing giving worse results

I am training a Decision tree regressor on the famous Boston House Price dataset. I read that tree based models are fairly immune to scaling so I tried to see practically. Before scaling I was getting ...
0
votes
1answer
46 views

Should I resample my dataset?

The dataset that I have is some text data consisting of path names. I am using TF-IDF vectorizer and decision trees. The classes in my dataset are severely imbalanced. There are a few big classes with ...
2
votes
1answer
37 views

Is it possible to do hard-coded decision tree on some variables and random forest / something on the remaining ones?

Is it possible to do hard-coded decision tree on some variables and random forest / something on the remaining ones? The situation seems that for some variables it's possible to draw strong empirical ...
1
vote
1answer
25 views

Computational vs intuitionistic or expert-based information gain in decision trees?

Computational vs intuitionistic or expert-based information gain in decision trees? This confuses me. Plenty of literature on how information gain can be used when it's calculated computationally. But ...
1
vote
5answers
205 views

What is the best way to train a model?

I am trying to train my model for sports predictions. The data frame is as a below given example: ...
0
votes
0answers
12 views

Uncertainty prediction in Gradient Boosted Tree based Quantile Regression

For an application, I am using a Gradient boosting Tree based quantile regression model (LightGBM, Catboot) to predict the 5th percentile of the target variable. The model predicts point estimates, ...
0
votes
1answer
33 views

Different values of mean absolute error when using GridSearchCV for max_leaf_nodes vs manually optimising max_leaf_nodes

I am trying out hyperparameter tuning vs manually selecting the best parameter (max_leaf_nodes) on a decision tree model with mean absolute error as the scoring. In ...
0
votes
1answer
50 views

Why does min-max scaler result in lower accuracy with regression tree?

I have a dataset that contains 7 features. Values are not too large. I trained scikit-learn's RandomForestRegressor for predicting the target variable. The $R^2$ ...
3
votes
1answer
48 views

Given M binary variables and R samples, what is the maximum number of leaves in a decision tree?

Given M binary variables and R samples, what is the maximum number of leaves in a decision tree? My first assumption was that the worst case would be a leaf for each sample, thus R leaves maximum. Am ...
2
votes
1answer
21 views

What if root of a such tree is pruned in xgboost?

Extreme Gradient Boosting stops to grow a tree if $\gamma$ is greater than impurity reduction given as eq (7) (see below) , what does happen if tree's root has a negative impurity? I think there is no ...
0
votes
1answer
51 views

How can we shorten our questionnaire to only ask the most informative question at each point?

Our product has an onboarding questionnaire which asks the same 58 questions (with numeric answers) to every new user. That’s a lot of questions, so we’d love to reduce the number of questions we ask ...
1
vote
0answers
9 views

Does anyone know of literature regarding a Neural Net boosted GBM?

For obvious reasons, most GBMs created in the private sector are tree boosted. Occasionally, one might want a linear boosted GBM so that the residual models collapse into a simple linear combination. ...
1
vote
1answer
43 views

Handling nominal category features in decision tree

I have been reading some stackoverflow questions on how to handle nominal features for decision tree (sklearn implementation). One of the answer states that : Using a OneHotEncoder is the only current ...
1
vote
1answer
159 views

XGBoost - Imputing Vs keeping NaN

What is the benefit of imputing numerical or categorical features when using DT methods such as XGBoost that can handle missing values? This question is mainly for when the values are missing not at ...
1
vote
0answers
8 views

Predict pixels in optdigits data set

I'm using this dataset : https://archive.ics.uci.edu/ml/datasets/optical+recognition+of+handwritten+digits a dataset that consists of 65 columns , the last column is the label for 10 classes i.e 0,1,2,...
1
vote
0answers
40 views

Model Tree M5 - Robustness to Data Quality Issues

I am currently investigating the M5 tree algorithm by Quinlan(1992) link here: https://sci2s.ugr.es/keel/pdf/algorithm/congreso/1992-Quinlan-AI.pdf An example of a linear regression model of the ...
0
votes
1answer
44 views

Tree based feature transformation

I'm new to data science field and interested in performing prediction using clickstream data. In Practical Lessons from Predicting Clicks on Ads at Facebook paper section 3.1, a method called Decision ...
0
votes
1answer
49 views

Best parameters to try while hyperparameter tuning in Decision Trees

I want to post prune my decision tree as it is overfitting, I can do this using cost complexity pruning by adjusting ccp_alphas parameters however this does not seem very intuitive to me. From my ...
0
votes
0answers
9 views

How to inform tree based methods about sequence of data

I am using tree-based methods for predictions, my series are such that they are usually high but then they slowly decrease to a lower level, I am giving the month and exogenous things as feature, but ...
0
votes
1answer
21 views

How does the random forest vote work?

I have a question. How is the voting done in random forests. I can't understand rationally, since we have a bootstrap sample drawn, and have built dection trees based on them, where is the new data ...
0
votes
0answers
9 views

Minimising inputs for decision tree predictions

It is common for decision trees with asymmetrical shapes to have leave nodes that come early. For example, the model can already generate a prediction if the answer to the first question is FALSE, ...
0
votes
0answers
15 views

Profiles classification

I've been working on a project where it consists of a dataset of profiles (50k rows) along with their position, age group and hobbies (200 columns). These features (except for the position) are graded ...
0
votes
1answer
40 views

Proof that Gini Impurity in a Decision Tree is Monotone Decreasing?

I asked this in a reply to an answer to another of my questions; but I think this merits its own question since I couldn't find an answer, and it's a pretty interesting question on its own. Suppose we ...
3
votes
1answer
133 views

How to create a confusion matrix for one node of a decision tree?

I am doing past papers for my data science exam and was curious about one of the questions. They ask us to create a confusion matrix by hand for one node of a decision tree. I understand how to create ...
0
votes
1answer
34 views

scikit learn target variable reversed (DecisionTreeClassifier)

I created a Decision Tree Classifier using sklearn, defined the target variable: ...
0
votes
1answer
20 views

How does sklearn random forest decide feature threshold at node splitting exactly?

Thinking of the RandomForestClassifier function in sklearn.ensemble, I understand that at each non-terminal node the algorithm: Randomly selects a subset of size max_features from the set of all ...

1
2 3 4 5
13