Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
3
votes
1answer
25 views

classification balanced target y [0,1] but imbalanced feature x [many 0 , few 1s] , maximize precision

I have a simple dataset with balanced target y (0 or 1) ,and imbalanced feature (many 0 , few 1's) I aim to get high precision (don't care about recall) I can get high precision of 0.53 if I just ...
2
votes
1answer
59 views

Decision tree Why is Gini index only used for binary choices?

I would like to understand why "Gini index operates on the categorical target variables in terms of “success” or “failure” and performs only binary split" ? Why it would not be possible to ...
1
vote
1answer
40 views

What does "S" in Shannon's entropy stands for?

I see many machine learning texts using the following notation to represent Shannon's entropy in classification/supervised learning contexts: $$ H(S) = \sum_{i \in Y}p_i \log(p_i) $$ Where $p_i$ is ...
1
vote
0answers
17 views

Is there a "tree-based-correlation" for tree-based algorithms?

Although correlated features are not a big issue when training tree-based models, they spoil model explainability. When several features correlate, sometimes they may be picked at random. Then their ...
1
vote
1answer
26 views

How different classifiers would perform on a particular data set

I am reading through and learning how different ML methods work on different types of data, but I have faced a data set that I am not sure how ML methods, such as decision tree, Naive Bayes, and KNN, ...
0
votes
2answers
27 views
1
vote
0answers
13 views

What kind of model to use to find drivers when data is aggregated and not on user level?

I have a website and have info from Google Analytics. So I can see the following "features": page url country device category (phone, desktop, etc.) Number of sessions Number of users: ...
1
vote
2answers
22 views

Multiple models have extreme differences during evaluation

My dataset has about 100k entries, 6 features, and the label is simple binary classification (about 65% zeros, 35% ones). When I train my dataset on different models: random forest, decision tree, ...
3
votes
0answers
32 views

Non-greedy decision tree / random forest implementation(s) in Python

The standard random forest is trained using a greedy approach for computational feasibility. However, there are a number of alternative methods/approaches such as "lookahead" or using ...
2
votes
0answers
20 views

Classifying short strings of text with additional context

I have a list of short strings each identifying a city. Misspellings are very common. The example below shows some of these short strings, along with the correct city they're supposed to match. ...
0
votes
0answers
6 views

Random Forests notebook collections to be used as benchmark

I'm trying to evaluate my modification of Decision Tree and Random Forests methods as compared to the standard distribution in libraries such as sklearn. I've searched the Kaggle, but did not find a ...
1
vote
1answer
42 views

How do I know if this model is overfitting?

This is my example R script for a decision tree: ...
6
votes
3answers
440 views

Is it possible to build ensemble models without a decision tree?

Is it possible to build ensemble models without a decision tree? I know that the description of ensembles itself suggests otherwise. However, I am really new to machine learning and all the ensemble ...
0
votes
1answer
32 views

Best Way to find the important features for the model [duplicate]

I have data with 245 Features and almost all of the features are categorical. I would like to know what will be the best approach to find the important features for training the model. I know I can ...
0
votes
1answer
42 views

What is the meaning of the Gini Index?

I'm studying random forest models, but I don't understand what the Gini index is and what it's for. Does anyone have any material on this or can give me an explanation? Thanks!
0
votes
1answer
11 views

Additional business rules in ensemble methods (RF, Boosted Trees)

How is it possible (if at all) to implement additional business constraints to an ensemble machine learning model, such as random forests or boosted trees? These additional business rules can be ...
1
vote
0answers
13 views

How are regression trees fit in gradient boosting for classification?

What I understood is that even gradient boosting for binary classification we use regression trees. The first value we calculate is constant = log(odds). For the rest of the trees we try to fit ...
1
vote
1answer
38 views

Scikit-learn's implementation of AdaBoost

I am trying to implement the AdaBoost algorithm in pure Python (or using NumPy if necessary)....
0
votes
1answer
23 views

Does hyperparameter tuning of Decision Tree then use it in Adaboost individually vs Simultaneously yield the same results?

So, my predicament here is as follows, I performed hyperparameter tuning on a standalone Decision Tree classifier, and I got the best results, now comes the turn of Standalone Adaboost, but here is ...
2
votes
2answers
31 views

confused on "real score" vs "decision value" in classification trees

I'm reading the guide to XGBoost and am confused about the distinction it draws between the scoring systems of decision trees and classification/regression trees. The paragraph I am hung up on is: A ...
1
vote
0answers
13 views

Classification for Ordinal labels - what tree-based methds can i use?

I have a label that has a natural ordering e.g. 0,1,2,3 where 0 is the worst activity measure and 3 is the best. For each label given by the model i need to also give the probability that it belongs ...
3
votes
2answers
51 views

Is it possible to 'group features' for a decision tree model?

At each node of a decision tree, we must choose a collection of features to split along. Suppose we know a priori that the features can be partitioned into subsets that are 'correlated', i.e. this ...
0
votes
0answers
20 views

Is it always that lower tree with higher bias but higher tree with higher varaince

When dealing with bias and variance trade-offs, I always hear that in tree models: shallow tree = high bias but low variance, deep tree = low bias but high variance. Someone may also quote from high ...
3
votes
1answer
58 views

Multi-target regression tree with additional constraint

I have a regression problem where I need to predict three dependent variables ($y$) based on a set of independent variables ($x$): $$ (y_1,y_2,y_3) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \...
0
votes
0answers
15 views

User Churn Rate analysis - Binary classification

I have a dataset which has the logs of user clicks. This is a trail version(2 months) of the software. Users can use a special feature during this trail period to improve their sales. The number of ...
0
votes
0answers
13 views

Is there any function of targets in binary classification decision tree

I was recently learning about decision tree and stumbled across a question which might be very silly but i am unable to understand it . That is if for a binary classification problem splits are used ...
0
votes
1answer
41 views

Orthogonal Decision Boundary in Decision Tree

I was reading the limitations of decision trees. One of them was that, for classification problems, decision trees produce only orthogonal decision boundaries. Could anyone please explain what an ...
2
votes
0answers
22 views

Building machine learning models whilst penalizing them for complexity

I come from a predictive modelling background, where it's common to use differential equations to model physical or chemical or biological processes. Commonly to avoid overfitting people use AIC and ...
0
votes
2answers
23 views

How to create classification decision trees on a dataset that has both numerical and categorical variables?

I am quite new to Data Science and learning things hands-on in the job. I am a fraud analyst and my job is to predict whether an application is fraudulent or not based on data. Before moving on to ...
0
votes
0answers
25 views

LightGBM boosting and bagging parameters

When training a gradient boosted decision tree model, I can use the LightGBM package to efficiently train my model. It's possible to define the hyperparameter search space with eg. ...
0
votes
0answers
31 views

Regression trees for extrapolating time series data

This is a regression problem that involves predicting the price of e.g. aluminum, oil, strawberries. I have hourly and half hourly data for the weather and up to 10 different socioeconomic variables (...
1
vote
1answer
24 views

Getting both results and probabilities running scikit learn random forest

I have a scikit learn RandomForestClassifier that returns 0s and 1s: ...
0
votes
0answers
37 views

Decision Tree taking too long to execute

I am training a Decision Tree Regressor on a relatively small data. The dimensions of my train and test sets are (34164, 10) and (8514, 10). Here is the relevant code: ...
0
votes
1answer
190 views

Combining heterogeneous numerical and text features

We want to solve a regression problem of the form "given two objects $x$ and $y$, predict their score (think about it as a similarity) $w(x,y)$". We have 2 types of features: For each ...
1
vote
1answer
43 views

Decision Trees and Categorical Feature Labelling

I am working on a decision tree model and trying to decide how best to handle categorical features. The features in my dataset are generally high in cardinality and I have found that ordinal labeling ...
0
votes
0answers
17 views

How to get variance for regression tree fit?

Suppose the true function is a tree such that: $$f(x)=\sum_{j=1}^{J}b_j I(x \in R_j)+e_i$$ where $b_j=E(y|x \in R_j)$ ,$E(e_i)=0$ and $R_j$ as terminal node. Suppose we got a fit for this tree via ...
0
votes
1answer
124 views

Is feature importance in XGBoost or in any other tree based method reliable?

This question is quite long, if you know how feature importance to tree based methods works i suggest you to skip to text below the image. Feature importance (FI) in tree based methods is given by ...
0
votes
1answer
34 views

Standardizing giving worse results

I am training a Decision tree regressor on the famous Boston House Price dataset. I read that tree based models are fairly immune to scaling so I tried to see practically. Before scaling I was getting ...
0
votes
1answer
47 views

Should I resample my dataset?

The dataset that I have is some text data consisting of path names. I am using TF-IDF vectorizer and decision trees. The classes in my dataset are severely imbalanced. There are a few big classes with ...
2
votes
1answer
40 views

Is it possible to do hard-coded decision tree on some variables and random forest / something on the remaining ones?

Is it possible to do hard-coded decision tree on some variables and random forest / something on the remaining ones? The situation seems that for some variables it's possible to draw strong empirical ...
1
vote
1answer
25 views

Computational vs intuitionistic or expert-based information gain in decision trees?

Computational vs intuitionistic or expert-based information gain in decision trees? This confuses me. Plenty of literature on how information gain can be used when it's calculated computationally. But ...
1
vote
5answers
211 views

What is the best way to train a model?

I am trying to train my model for sports predictions. The data frame is as a below given example: ...
0
votes
0answers
14 views

Uncertainty prediction in Gradient Boosted Tree based Quantile Regression

For an application, I am using a Gradient boosting Tree based quantile regression model (LightGBM, Catboot) to predict the 5th percentile of the target variable. The model predicts point estimates, ...
0
votes
1answer
44 views

Different values of mean absolute error when using GridSearchCV for max_leaf_nodes vs manually optimising max_leaf_nodes

I am trying out hyperparameter tuning vs manually selecting the best parameter (max_leaf_nodes) on a decision tree model with mean absolute error as the scoring. In ...
0
votes
1answer
64 views

Why does min-max scaler result in lower accuracy with regression tree?

I have a dataset that contains 7 features. Values are not too large. I trained scikit-learn's RandomForestRegressor for predicting the target variable. The $R^2$ ...
3
votes
1answer
54 views

Given M binary variables and R samples, what is the maximum number of leaves in a decision tree?

Given M binary variables and R samples, what is the maximum number of leaves in a decision tree? My first assumption was that the worst case would be a leaf for each sample, thus R leaves maximum. Am ...
2
votes
1answer
22 views

What if root of a such tree is pruned in xgboost?

Extreme Gradient Boosting stops to grow a tree if $\gamma$ is greater than impurity reduction given as eq (7) (see below) , what does happen if tree's root has a negative impurity? I think there is no ...
0
votes
1answer
54 views

How can we shorten our questionnaire to only ask the most informative question at each point?

Our product has an onboarding questionnaire which asks the same 58 questions (with numeric answers) to every new user. That’s a lot of questions, so we’d love to reduce the number of questions we ask ...
1
vote
0answers
9 views

Does anyone know of literature regarding a Neural Net boosted GBM?

For obvious reasons, most GBMs created in the private sector are tree boosted. Occasionally, one might want a linear boosted GBM so that the residual models collapse into a simple linear combination. ...
1
vote
1answer
58 views

Handling nominal category features in decision tree

I have been reading some stackoverflow questions on how to handle nominal features for decision tree (sklearn implementation). One of the answer states that : Using a OneHotEncoder is the only current ...

1
2 3 4 5
13