Questions tagged [bagging]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Multiclass classification oob error

Im implementing a random forrest for a 6 class classification and witnessing a strange phenomenon. I have 10 percent of my set sectioned out as a pseudo validation set. Im training 50 percent of the ...
0
votes
0answers
11 views

xgboost performance

XGBoostRegressor is not performing better than AdaBoostRegressor for the same set of parameters for some reason. Since my ...
1
vote
1answer
18 views

Why can't we sample without replacement for each tree in a random forest if the subsample size is large enough?

Usually if we have $n$ observations, for each tree with form a bootstrapped subsample of size $n$ with replacement. On googling it one common explanation I've seen is that with replacement sampling is ...
4
votes
1answer
83 views

Can Boosting and Bagging be applied to heterogeneous algorithms?

Stacking can be achieved with heterogeneous algorithms such as RF, SVM and KNN. However, can such heterogeneously be achieved in Bagging or Boosting? For example, in Boosting, instead of using RF in ...
0
votes
1answer
25 views

Difference between bagging and pasting?

I found the definition: ...
0
votes
0answers
7 views

How to apply feature selection in cross validated bagging

Normally in cross validation decision tree, feature selection will occur with training data but in bagging ensemble the training data is bootstrapped. How can I apply feature selection in cross ...
1
vote
1answer
26 views

Base model in ensemble learning

I've been doing some research on ensemble learning and read that for base models, model with high variance are often recommended (can't remember which book I read this from exactly). But, it seems ...
2
votes
1answer
24 views

bagging vs. pasting in ensemble learning

I am bit confused about two concepts. From my understanding Bagging is when each data is replaced after each choice. so for example for each subset of data you pick one from population, replace it ...
0
votes
1answer
21 views

Bagging Base models

If bagging reduces overfitting than the general statement that base learners of ensemble models should have high bias and low variance(that is should be undefiting) wrong?
0
votes
1answer
50 views

Why the accuracy of my bagging model heavily affected by random state? [closed]

The accuracy of my bagging decision tree model reach up to 97% when I set the random seed=5 but the accuracy reduce to only 92% when I set random seed=0. Can someone explain why the huge gap and ...
1
vote
1answer
85 views

Counting the number of trainable parameters in a gradient boosted tree

I recently ran the gradient boosted tree regressor using scikit-learn via: GradientBoostingRegressor() This model depends on the following hyperparameters: Estimators ($N_1$) Min Samples Leaf ($N_2$...
2
votes
1answer
28 views

Can I do bagging method as improvement technique to decision tree in research?

Bagging use decision tree as base classifier. I want to use bagging with decision tree(c4.5) as base as the method that improve decision tree(c4.5) in my research that solve problem overfitting. Is ...
3
votes
0answers
69 views

Difference Bagging and Bootstrap aggregating

Bootstrap belongs to Efron. Tibshirani wrote a book about that in reference to Efron. Bootstrap process for estimating the standard error of statistic s(x). B bootstrap sample are generatied from ...
2
votes
1answer
55 views

How does bagging help reduce the variance

I learned that bagging helps reduce variance by averaging but I couldn't understand this. Can someone explain this intuitively?
1
vote
0answers
67 views

Can bagging ensemble consist of heterogeneous base models?

Bagging or bootstrap aggregation seems to make sense for time series forecasting using an ensemble because bagging randomizes subsets of the data with replacement. However, I've only seen bagging used ...
2
votes
1answer
36 views

Random Forest Stacking Experiment for Imbalanced Data-set Problem

In order to solve a Imbalanced Dataset Problem, I experimented with Random Forest in the given manner (Somewhat inspired by Deep-Learning) Trained a Random Forest which will take in the input data ...
3
votes
1answer
129 views

Bagging vs pasting in ensemble learning

This is a citation from "Hands-on machine learning with Scikit-Learn, Keras and TensorFlow" by Aurelien Geron: "Bootstrapping introduces a bit more diversity in the subsets that each predictor is ...
6
votes
1answer
549 views

Boosting with highly correlated features

I have a conceptual question. My understanding is, that Random Forest can be applied even when features are (highly) correlated. This is because with bagging, the influence of few highly correlated ...