Questions tagged [adaboost]
The adaboost tag has no usage guidance.
26
questions
0
votes
0
answers
10
views
Scikit-learn's SAMME AdaBoost error fraction implementation
I am taking a look at scikit-learn's discrete SAMME implementation and came across the following logic for computing the weighted error fraction.
...
- 11
0
votes
1
answer
62
views
How does Adaboost reassure us that It'll do better after each iteration?
From what I know, AdaBoost works by concat-ing a weak classifier(ussually a one-level decision tree) to the previous linear combination of other weak classifiers to improve its accuracy after each ...
0
votes
2
answers
89
views
How is a single classifier fitted on AdaBoost?
The AdaBoost algorithm is:
My trouble is how the classifier $G_m(x)$ is trained, What does mean a classifier to be trained using weights $w_i$? Is it to fit classifier through $\{w_i,y_i\}_{i=1}^{N}$?...
- 123
1
vote
1
answer
143
views
Scikit-learn's implementation of AdaBoost
I am trying to implement the AdaBoost algorithm in pure Python (or using NumPy if necessary)....
- 177
0
votes
1
answer
108
views
Does hyperparameter tuning of Decision Tree then use it in Adaboost individually vs Simultaneously yield the same results?
So, my predicament here is as follows, I performed hyperparameter tuning on a standalone Decision Tree classifier, and I got the best results, now comes the turn of Standalone Adaboost, but here is ...
- 23
2
votes
1
answer
170
views
Why does classifier (XGBoost) “after PCA” runtime increase compared to “before PCA”
The short version: I am trying to compare different classifiers for a certain dataset from kaggle, and am trying to also compare these classifiers between before using PCA (form sklearn) to after ...
- 41
1
vote
1
answer
34
views
Evaluating optimal values for depth of tree
I'm studying the performance of an AdaBoost model and I wonder how it performs in regard to the depth of the trees.
Here's the accuracy for the model with a depth of 1
and here with a depth of 3
...
- 540
3
votes
2
answers
406
views
How to use a set of pre-defined classifiers in Adaboost?
Suppose there are some classifiers as follows:
...
- 201
-1
votes
1
answer
94
views
Forecasting: Multiple Linear Regression (OLS) outperforms Random Forests / Gradient Boosting / AdaBoost
I'm using different forecasting methods on a dataset to try and compare the accuracy of these methods.
For some reason, multiple linear regression (OLS) is outperforming RF, GB and AdaBoost when ...
- 11
5
votes
1
answer
84
views
Is the way to combine weak learners in AdaBoost for regression arbitrary?
I'm reading about how variants of boosting combine weak learners into final predication. The case I'm consider is regression.
In paper Improving Regressors using Boosting Techniques, the final ...
- 175
1
vote
1
answer
834
views
How Adaboost calculates error for each weak learner in training?
I am studying the Adaboost classification algorithm because i would like to implement it from scratch. I understand how it works, but i am not able to understand where some steps are placed.
I will ...
- 395
1
vote
1
answer
78
views
Adaboost with other classifier fitting
There is the opportunity to fit decision trees with other decision trees. For example:
...
- 329
1
vote
2
answers
697
views
Does gradient boosting algorithm error always decrease faster and lower on training data?
I am building another XGBoost model and I'm really trying not to overfit the data. I split my data into train and test set and fit the model with early stopping based on the test-set error which ...
- 182
1
vote
0
answers
86
views
AdaBoost.R2 learning rate from scikit learn
AdaBoost.R2 (regression), is presented in the paper "improving regressors with boosting techniques" from Drucker and is freely available on Scholar.
The implementation of regression for ...
1
vote
1
answer
80
views
Why Adaboost SAMME needs f to be estimable?
I am trying to understand the mathematics behind SAMME AdaBoost:
At some stage, the paper adds a constraint for f to be estimable:
I do not understand why this is ...
- 45
1
vote
1
answer
107
views
Explanation on some steps of AdaBoost.R2
I am trying to understand AdaBoost.R2 in order to implement it and apply it to a regression problem. In this circumstances I need to understand it perfectly, however there's some step i don't really ...
2
votes
0
answers
28
views
Understanding additive function approximation or Understanding matching pursuit
I am trying to read Greedy function approximation: A gradient boosting machine.
On page 4 (it is marked as page 1192) under 3. Finite data the author tells how the function approximation approach ...
- 821
1
vote
0
answers
78
views
AdaBoost decision_function() outputs in binary classification with sklearn
As I understand it based on some study of the source code, I would expect, when using AdaBoost, that values obtained by calling decision_function() would be bounded between -1 and 1. This is because ...
- 111
1
vote
0
answers
87
views
Every machine learing model i build, always predict wrongly almost the same samples. (Random forest, XGBoost, AdaBoost)
First of all, I'd like to apologize for any spelling or grammar mistakes.
I'm having a problem using R for a classification problem. My dataset contains ~300.000 genomic data, and the features are ...
1
vote
1
answer
136
views
Formula to calculate confidence value in Adaboost
I am coding an AdaBoostClassifier with the two class variant of SAMME algorithm. Here is the code.
def I(flag):
return 1 if flag else 0
...
2
votes
1
answer
2k
views
Decreasing n_estimators is increasing accuracy in AdaBoost?
I was exploring the AdaBoost classifier in sklearn. This is the plot of the dataset.
(X,Y are the predictor columns and the color is the label)
As you can see there are exactly 16 points in either ...
1
vote
0
answers
133
views
Interpreting Adaboost model results
I'm trying to get a better grasp of model interpretability using many different kinds of models for a binary classification problem.
Quick note: By interpretability in this case, what I mean is ...
- 111
1
vote
1
answer
134
views
what are the steps in adaboosting?
I went through adaboost tutorial and below are my simplified understanding:
Sample weight of equal value is given to all sample in dataset.
Stumps are created which uses only one feature from data ...
- 13
1
vote
0
answers
91
views
Sklearn Decision Tree as weak learner in Adaboost not working properly
I'm trying to implement Adaboost algorithm with sklearn decision tree as the Weak Learner - at each step I want to choose one feature with one threshold to classify all samples.
I have 1400 long ...
2
votes
0
answers
103
views
AdaBoost vs Gradient Boost [duplicate]
What is the difference? Under which criteria should each type of boost be used?
What is the theory behind each of these methods?
9
votes
0
answers
2k
views
AdaBoost implementation and tuning for high dimensional feature space in R
I am trying to implement the AdaBoost.M1 algorithm (trees as base-learners) to a data set with a large feature space (~ 20.000 features) and ~ 100 samples in R. ...
- 91