Questions tagged [decision-trees]

A decision tree is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. It is one way to display an algorithm.

Filter by
Sorted by
Tagged with
-1
votes
1answer
12 views

could not convert string to float

i have a string in my dataset trying to apply random forest regression getting an error could not convert string to float code is: Importing the libraries import numpy as np import matplotlib....
1
vote
1answer
11 views

Error rate of AdaBoost weak learner always bigger than 0.5?

As far as i understand, weak learners of AdaBoost should never yield a error rate > 0.5 After training one, i only receive error rates above 0.5. How is that even possible? The AdaBoost Tree still ...
1
vote
2answers
27 views

class_weight on sklearn's DecisionTreeClassifier

Can class_weight='balanced' on scikit-learn's DecisionTreeClassifier be interpreted as having identical duplicate data points for the minority classes? I know that doesn't work that way, class_weight ...
1
vote
0answers
20 views

Continuous decision trees using logistic functions

Decision tree functions are discontinuous functions of the predictors. Have continuous decision trees with smooth transitions been studied? For example, a decision tree in two variables ...
0
votes
0answers
18 views

Can linear classifiers be used at each node of a decision tree instead of the lines parallel to any one of the axes?

I am relatively new to AI/ML. I came across this question while reading some content on ML. Would be of great help if anyone can answer this
0
votes
0answers
6 views

Combining decision trees and neural networks for classifying text with metadata . How to combine and train?

I have a multi-label classification problem where the input consist of free text, with metadata such as categories (from a fixed, limited set) associated with each text. The output consist of a set of ...
2
votes
2answers
50 views

Can a decision tree learn to solve a xOR problem?

I have read online that decision trees can solve xOR type problems, as shown in images (xOR problem: 1) and (Possible solution as decision tree: 2). My question is how can a decision tree learn to ...
1
vote
1answer
34 views

random forest classifier - impact of small n_estimator and repeated training

trying to have a better understanding of random forest algorithm here. With the same training and holdout datasets, I tried two things here: Set a small n_estimator (10), train on my training dataset ...
0
votes
1answer
19 views

How to tell a boosting model that 2 features are related and should not be interpreted stand-alone?

I am using XGBoost for a machine learning model that learns from tabular data. XGBoost uses boosting method on decision trees. When I look at the decision-making logic of decision trees, I notice the ...
0
votes
1answer
20 views

Does Decision tree classifier calculate entropies before transforming categorical features using OneHotEncoder or transformation should be done

I am new to machine learning, and I've got to the point to drop out from it as online tutorials are pretty confusing as well. Entropy and Decision trees One of confusing tutorials was as the ...
2
votes
0answers
35 views

Why decision tree algorithm performs better in small dataset in compare to other classification algorithm?

I read in some paper that decision tree algorithms have better result in small dataset like(400-600 records) when compared to other classification algorithms like SVM, Naive Bayes and KNN. I want to ...
1
vote
2answers
104 views

How to find “regions” with high purity

I am applying a ML model (LGBM binary classifier) to data and would now like to identify the part of data where I have a low ratio of false-negatives (false-postives are not such a problem) and as ...
0
votes
1answer
22 views

Is it necessary to convert labels in string to integer for scikit_learn and xgboost?

I have a tabular data with labels that are in string. I will feed the data to decision trees in scikit_learn and XGBoost classifier. Is it necessary to convert the labels in string to integers for ...
0
votes
1answer
11 views

Is it necessary to normalise data for XGBoost?

MinMaxScaler in scikit_learn is used for data normalization (a.k.a feature scaling). Data normalisation is not necessary for decision trees. Since XGBoost is based on decision trees, is it necessary ...
1
vote
0answers
19 views

What is the best approach to train a multi-category regression model?

What is the best approach to train a multi-category regression model (using XBoost decision trees ensemble)? What are the ups and downs of each one? For example, if I want to train a model to predict ...
1
vote
1answer
22 views

Can a decision in a node of a decision tree be based on comparison between 2 columns of the dataset?

Assume the features in the dataframe are columns - A,B,C and my target is Y Can my decision tree have a decision node which looks for say, ...
0
votes
0answers
12 views

What can be concluded about a dataset if Extra Trees is the one algo that gives the best accuracy?

Basically the title. It seems to me that ET is a specific type of algo, with a very low variance (presumably high bias then?). What does that say about my dataset? My intuition is: too few samples, ...
1
vote
1answer
19 views

Tree based method are robust against low probability feature space zones when using ML general interpretability methods?

I have this intuition but I'm not able to verify it. There are a lot of techniques to understand the effect of single features in ML models. Some take inspiration from counterfactual frameworks, like ...
3
votes
1answer
39 views

Validation Curve Interpretations for Decision Tree

I'm working on a machine learning class, and we're using supervised learning right now, starting with decision trees. I'm using the UCI Credit Card dataset (whether or not certain people will default ...
1
vote
0answers
30 views

Would writing a decision tree algorithm in Pytorch or Tensorflow be faster than with Numpy?

Since these libraries can turn CPU arrays into GPU tensors, could you parallelize (and therefore accelerate) the calculations for a decision tree? I am considering making a decision tree class written ...
0
votes
0answers
15 views

Decision Tree with ordinal target variable - Which algorithm?

I want to construct a decision tree as part of my MA thesis. I have several variables which are either nominal or ordinal. The target variable (i.e. the dependent variable) is also ordinal. I am ...
0
votes
0answers
9 views

Are Decission trees with depth=1 also non linear?

Suppose: Root node= x>100 left node= True right node= Right Can anyone explain if this is a linear or non linear model with a reason? The answers to this ...
0
votes
0answers
13 views

Should I convert noncontinuous numerical values to categorical features?

I'm working with very sparse matrixes and have several noncontinuous numerical fields. Are these values better utilized as continuous (prevent from increasing sparseness) or as categorical features? ...
0
votes
2answers
61 views

Decision Tree split error

During the split function of the decision tree, I am getting an assertion error stating: Error > Assert: type(C) == dict I couldn't find any errors in the ...
1
vote
0answers
47 views

what do the percentages in the leaves of a decision tree represent?

in figure B), there are leaves (gray boxes) with 3 values, for example, the leftmost leaf has 19.3 (28/8.7%) as its values, the 3 values are (19.3, 28, and 8.7%). 19.3 is the average value of the ...
1
vote
0answers
16 views

Weighted Entropy (or) Conditional Entropy calculation error

I'm getting an assertion error for the test case: Y = np.array([0.,1.,0.,1.]), X = np.array([1.,1.,4.,4.]) The code executes when assert np.allclose(ce,0.,atol =1e-...
0
votes
3answers
96 views

Ordinal features to decision tree in Python

I have a data set with ordinal features.Each feature might have 6 to 7 levels. Based on my search for R if you have ordinal data, rpart treats ordinal and nominal differently. https://stats....
1
vote
1answer
21 views

Regression Trees - Splitting and decision rules

I understand that a regression tree is built by splitting a node, such that the MSE for the label/output variable is minimized in each of the two resulting nodes. I have two questions about this: 1.)...
-1
votes
1answer
224 views

Conditional entropy calculation in python, H(Y|X)

Input X: A numpy array whose size gives the number of instances. X contains each instance's attribute value. Y: A numpy array which contains each instance's corresponding target label. Output : ...
0
votes
1answer
35 views

Error using decision tree regressor [closed]

I'm new to data science , while i'm implementing decision tree. I'm facing the following error. Where i went wrong; Sample data in csv is: ...
0
votes
1answer
17 views

What are the factors to consider when setting the depth of a decision tree?

In scikit learn, one of the parameters to set when instantiating a decision tree is the maximum depth. What are the factors to consider when setting the depth of a decision tree? Does larger depth ...
0
votes
1answer
23 views

How will a decision tree cope if it cannot find a suitable feature to choose as root node from which to split further?

When I look at decision trees, they start with a root node choosing the most suitable feature from which to split further. What if the decision tree is unable to find the most suitable feature from ...
4
votes
1answer
106 views

How to extract trees in XGBoost?

I want to extract each tree so that I can feed it with any data, and see the output. ...
2
votes
2answers
42 views

Gradient boosting how can accuracy increase when we lower the depth of tree?

What I don't understand about gradient boost is, doesn't lowering height of the tree means we use fewer features in our model? From my model I get the highest accuracy when the depth is one. Meaning ...
3
votes
2answers
40 views

Decision Trees Should We Discard Low Importance Features?

I just started to work with feature selection. Let's say I have a decision tree model. I get its feature importances by tree.feature_importances_. In my model out ...
0
votes
4answers
41 views

How to know if my Decision tree model is good or bad?

I built a decision tree model and am not sure if it is good or bad. Could you help to evaluate my model? My code: ...
2
votes
2answers
67 views

Calculate future GDP % using machine learning

I need to estimate the GDP % of a country three years into the future, based on historic data. I have 30+ years of the following monthly data that includes features such as inflation and unemployment ...
0
votes
1answer
54 views

Decision tree classifier prediction changes from one run of the model to the next

I'm running a very basic gender ['male', 'female'] classifier using the sklearn DecisionTreeClassifier based on ...
2
votes
2answers
55 views

Decision Trees - how does split for categorical features happen?

A decision tree, while performing recursive binary splitting, selects an independent variable (say $X_j$) and a threshold (say $t$) such that the predictor space is split into regions {$X|X_j < t$} ...
1
vote
1answer
12 views

clarification on splitting individual trees in extra trees?

So I am a beginner in machine learning and just started learning about random trees in this article here. When it talks about tuning the hyperparameter K, I'm a bit confused as to how it works. It ...
0
votes
1answer
34 views

CART algorithm (Classification and regression trees) question

So this is taken from an exam I just did. I'd like to know if there are any instances same as in the image where the CART algorithm could use a negative alpha and thus encourage a larger tree? Or does ...
1
vote
1answer
23 views

when to use certain metrics for splitting decision trees?

So I just very recently learned about decision trees, and the different metrics for determining the best split when training the tree. I cannot seem to be able to find anything on which metric to use ...
1
vote
0answers
12 views

Gradient Boosting Partial Dependency Plot

I have been trying to generate a partial development plot using gradient boosting. The Plot looks like as below. My question is why the plot shows two or three steps rather than several broken ...
0
votes
1answer
18 views

How to interpret fit from regression (decision) tree which has used 0 variables

I have fit a regression tree to my dataset and the output from summary(tree1) is as follows: ...
2
votes
1answer
118 views

How does class_weight work in Decision Tree

The scikit-learn implementation of DecisionTreeClassifier has a parameter as class_weight. ...
1
vote
2answers
96 views

How does Decision Tree with Gini Impurity Calculate Root Node?

I couldn't figure out how it selected the root node with with <=7.5 and it's gini impurity is 0.45 but I tried to manually ...
0
votes
1answer
14 views

How should a decision tree handle an attribute that can be anything?

Say I have AttributeA that can take values A1, A2, A3, AttributeB that can take values B1, B2, B3, etc. and I know ahead of time that my classification table looks like AttributeA | AttributeB | ...
0
votes
2answers
42 views

Can we use DecisionTreeClassifier of sklearn for continuous target variable?

I have a continuous target variable named "quality" which ranges from 0 to 10. Also I have 11 input variables in my dataset. When I'm building my model using DecisionTreeClassifier() of sklearn then ...
2
votes
1answer
43 views

Is it possible to ensure that all classes are represented in the output of a scikit-learn decision tree?

I am working with an ordinal classification problem with six ordered classes and I want to compare a neural network classifier with a baseline classifier that is as simple and parameter-free as ...
0
votes
0answers
32 views

Find similarity between 2 sample data points using Random Forest Classifier

Let’s assume that you trained a Random Forest model with 10 estimators on a dataset and passed 2 sample data points (S1 and S2) through each of the trees in the forest. You get to see the leaf node ...