Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [adaboost]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
2 answers
39 views

How is a single classifier fitted on AdaBoost?

The AdaBoost algorithm is: My trouble is how the classifier $G_m(x)$ is trained, What does mean a classifier to be trained using weights $w_i$? Is it to fit classifier through $\{w_i,y_i\}_{i=1}^{N}$?...
user avatar
1 vote
1 answer
62 views

Scikit-learn's implementation of AdaBoost

I am trying to implement the AdaBoost algorithm in pure Python (or using NumPy if necessary)....
user avatar
0 votes
1 answer
47 views

Does hyperparameter tuning of Decision Tree then use it in Adaboost individually vs Simultaneously yield the same results?

So, my predicament here is as follows, I performed hyperparameter tuning on a standalone Decision Tree classifier, and I got the best results, now comes the turn of Standalone Adaboost, but here is ...
user avatar
0 votes
0 answers
12 views

Train and test data fixed during boosting?

I have question about boosting algorithm. I know that boosting is a sequential process and it gives high weight to misclassification of previous model. Then, its' train and test data are fixed through ...
user avatar
  • 1
2 votes
1 answer
112 views

Why does classifier (XGBoost) “after PCA” runtime increase compared to “before PCA”

The short version: I am trying to compare different classifiers for a certain dataset from kaggle, and am trying to also compare these classifiers between before using PCA (form sklearn) to after ...
user avatar
1 vote
1 answer
25 views

Evaluating optimal values for depth of tree

I'm studying the performance of an AdaBoost model and I wonder how it performs in regard to the depth of the trees. Here's the accuracy for the model with a depth of 1 and here with a depth of 3 ...
user avatar
  • 480
3 votes
2 answers
154 views

How to use a set of pre-defined classifiers in Adaboost?

Suppose there are some classifiers as follows: ...
user avatar
  • 181
-1 votes
1 answer
51 views

Forecasting: Multiple Linear Regression (OLS) outperforms Random Forests / Gradient Boosting / AdaBoost

I'm using different forecasting methods on a dataset to try and compare the accuracy of these methods. For some reason, multiple linear regression (OLS) is outperforming RF, GB and AdaBoost when ...
user avatar
  • 11
5 votes
1 answer
45 views

Is the way to combine weak learners in AdaBoost for regression arbitrary?

I'm reading about how variants of boosting combine weak learners into final predication. The case I'm consider is regression. In paper Improving Regressors using Boosting Techniques, the final ...
user avatar
  • 175
1 vote
1 answer
44 views

How Adaboost calculates error for each weak learner in training?

I am studying the Adaboost classification algorithm because i would like to implement it from scratch. I understand how it works, but i am not able to understand where some steps are placed. I will ...
user avatar
1 vote
1 answer
63 views

Adaboost with other classifier fitting

There is the opportunity to fit decision trees with other decision trees. For example: ...
user avatar
  • 329
1 vote
2 answers
384 views

Does gradient boosting algorithm error always decrease faster and lower on training data?

I am building another XGBoost model and I'm really trying not to overfit the data. I split my data into train and test set and fit the model with early stopping based on the test-set error which ...
user avatar
  • 203
1 vote
0 answers
52 views

AdaBoost.R2 learning rate from scikit learn

AdaBoost.R2 (regression), is presented in the paper "improving regressors with boosting techniques" from Drucker and is freely available on Scholar. The implementation of regression for ...
user avatar
1 vote
1 answer
59 views

Why Adaboost SAMME needs f to be estimable?

I am trying to understand the mathematics behind SAMME AdaBoost: At some stage, the paper adds a constraint for f to be estimable: I do not understand why this is ...
user avatar
1 vote
1 answer
30 views

Explanation on some steps of AdaBoost.R2

I am trying to understand AdaBoost.R2 in order to implement it and apply it to a regression problem. In this circumstances I need to understand it perfectly, however there's some step i don't really ...
user avatar
2 votes
0 answers
21 views

Understanding additive function approximation or Understanding matching pursuit

I am trying to read Greedy function approximation: A gradient boosting machine. On page 4 (it is marked as page 1192) under 3. Finite data the author tells how the function approximation approach ...
user avatar
1 vote
0 answers
42 views

AdaBoost decision_function() outputs in binary classification with sklearn

As I understand it based on some study of the source code, I would expect, when using AdaBoost, that values obtained by calling decision_function() would be bounded between -1 and 1. This is because ...
user avatar
  • 11
1 vote
0 answers
80 views

Every machine learing model i build, always predict wrongly almost the same samples. (Random forest, XGBoost, AdaBoost)

First of all, I'd like to apologize for any spelling or grammar mistakes. I'm having a problem using R for a classification problem. My dataset contains ~300.000 genomic data, and the features are ...
user avatar
1 vote
1 answer
94 views

Formula to calculate confidence value in Adaboost

I am coding an AdaBoostClassifier with the two class variant of SAMME algorithm. Here is the code. def I(flag): return 1 if flag else 0 ...
user avatar
2 votes
1 answer
1k views

Decreasing n_estimators is increasing accuracy in AdaBoost?

I was exploring the AdaBoost classifier in sklearn. This is the plot of the dataset. (X,Y are the predictor columns and the color is the label) As you can see there are exactly 16 points in either ...
user avatar
1 vote
0 answers
64 views

Interpreting Adaboost model results

I'm trying to get a better grasp of model interpretability using many different kinds of models for a binary classification problem. Quick note: By interpretability in this case, what I mean is ...
user avatar
1 vote
1 answer
120 views

what are the steps in adaboosting?

I went through adaboost tutorial and below are my simplified understanding: Sample weight of equal value is given to all sample in dataset. Stumps are created which uses only one feature from data ...
user avatar
1 vote
0 answers
53 views

Sklearn Decision Tree as weak learner in Adaboost not working properly

I'm trying to implement Adaboost algorithm with sklearn decision tree as the Weak Learner - at each step I want to choose one feature with one threshold to classify all samples. I have 1400 long ...
user avatar
2 votes
0 answers
83 views

AdaBoost vs Gradient Boost [duplicate]

What is the difference? Under which criteria should each type of boost be used? What is the theory behind each of these methods?
user avatar
8 votes
0 answers
2k views

AdaBoost implementation and tuning for high dimensional feature space in R

I am trying to implement the AdaBoost.M1 algorithm (trees as base-learners) to a data set with a large feature space (~ 20.000 features) and ~ 100 samples in R. ...
user avatar
  • 81