Skip to main content

Questions tagged [bagging]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
1 answer
26 views

Average training instances sampled with bagging

The book Hands-On Machine Learning has a section on Out-of-Bag Evaluation related to Decision Trees, where it's stated that, By default a BaggingClassifier samples m training instances with ...
Sahil Gupta's user avatar
0 votes
0 answers
13 views

i am facing problem in combining the forecasts generated by multiple stacked GRU models

first i have used CEEMDAN, to decompose multi-variate data into different IMFs, after that for each IMF stacked GRU model is trained, which gives results for each IMFs, for the final predictions i ...
Poonam Dhaka's user avatar
0 votes
2 answers
93 views

How a Random forest "learns" or How loss (objective function value) is propagated back so that a random forest can "Improve"?

Every Blog and Youtube video talks about the same steps: Choose that you have to build N number of tree and do the task 2-5 ...
Deshwal's user avatar
  • 323
1 vote
1 answer
36 views

Can the product of tree regressions be represented by a single tree?

Assume that we have two separate tree regressions. I'm interested in understanding whether the product of tree regressions can be represented by a single tree. Would this be possible?
TFT's user avatar
  • 135
1 vote
1 answer
44 views

What are valid measures for reporting k-fold score in the case of confusion-matrix?

I know when model is made to predict a float value, a common approach to report the models validation is using k-fold technique and calculating the average of all folds accuracy (here is a similar ...
morteza's user avatar
  • 11
0 votes
1 answer
28 views

Where should I stop training if I want to bag models

Let's say I have a clear case of overfitting where my loss curves look like this (x axis are iterations): Now I would like to try bagging to reduce the variance, where should I stop models training? ...
dzi's user avatar
  • 111
2 votes
1 answer
285 views

Random LightGBM Forest

I'm not completly sure about the bias/variance of boosted decision trees (LightGBM especially), thus I wonder if we generally would expect a performance boost by creating an ensemble of multiple ...
CutePoison's user avatar
2 votes
1 answer
1k views

Why can't we sample without replacement for each tree in a random forest if the subsample size is large enough?

Usually if we have $n$ observations, for each tree with form a bootstrapped subsample of size $n$ with replacement. On googling it one common explanation I've seen is that with replacement sampling is ...
user9343456's user avatar
5 votes
1 answer
612 views

Can Boosting and Bagging be applied to heterogeneous algorithms?

Stacking can be achieved with heterogeneous algorithms such as RF, SVM and KNN. However, can such heterogeneously be achieved in Bagging or Boosting? For example, in Boosting, instead of using RF in ...
Ahmad Bilal's user avatar
0 votes
1 answer
2k views

Difference between bagging and pasting?

I found the definition: ...
good_evening's user avatar
1 vote
1 answer
102 views

Base model in ensemble learning

I've been doing some research on ensemble learning and read that for base models, model with high variance are often recommended (can't remember which book I read this from exactly). But, it seems ...
haneulkim's user avatar
  • 469
2 votes
1 answer
220 views

bagging vs. pasting in ensemble learning

I am bit confused about two concepts. From my understanding Bagging is when each data is replaced after each choice. so for example for each subset of data you pick one from population, replace it ...
haneulkim's user avatar
  • 469
0 votes
1 answer
46 views

Bagging Base models

If bagging reduces overfitting than the general statement that base learners of ensemble models should have high bias and low variance(that is should be undefiting) wrong?
Aman Oswal's user avatar
1 vote
1 answer
394 views

Why the accuracy of my bagging model heavily affected by random state? [closed]

The accuracy of my bagging decision tree model reach up to 97% when I set the random seed=5 but the accuracy reduce to only 92% when I set random seed=0. Can someone explain why the huge gap and ...
Farrah 1234's user avatar
2 votes
1 answer
1k views

Counting the number of trainable parameters in a gradient boosted tree

I recently ran the gradient boosted tree regressor using scikit-learn via: GradientBoostingRegressor() This model depends on the following hyperparameters: Estimators ($N_1$) Min Samples Leaf ($N_2$...
ABIM's user avatar
  • 123
2 votes
1 answer
50 views

Can I do bagging method as improvement technique to decision tree in research?

Bagging use decision tree as base classifier. I want to use bagging with decision tree(c4.5) as base as the method that improve decision tree(c4.5) in my research that solve problem overfitting. Is ...
Farrah 1234's user avatar
3 votes
0 answers
139 views

Difference Bagging and Bootstrap aggregating

Bootstrap belongs to Efron. Tibshirani wrote a book about that in reference to Efron. Bootstrap process for estimating the standard error of statistic s(x). B bootstrap sample are generatied from ...
martin's user avatar
  • 329
2 votes
1 answer
187 views

How does bagging help reduce the variance

I learned that bagging helps reduce variance by averaging but I couldn't understand this. Can someone explain this intuitively?
Bhuwan Bhatt's user avatar
1 vote
0 answers
131 views

Can bagging ensemble consist of heterogeneous base models?

Bagging or bootstrap aggregation seems to make sense for time series forecasting using an ensemble because bagging randomizes subsets of the data with replacement. However, I've only seen bagging used ...
develarist's user avatar
2 votes
1 answer
98 views

Random Forest Stacking Experiment for Imbalanced Data-set Problem

In order to solve a Imbalanced Dataset Problem, I experimented with Random Forest in the given manner (Somewhat inspired by Deep-Learning) Trained a Random Forest which will take in the input data ...
Aman Raparia's user avatar
3 votes
2 answers
547 views

Bagging vs pasting in ensemble learning

This is a citation from "Hands-on machine learning with Scikit-Learn, Keras and TensorFlow" by Aurelien Geron: "Bootstrapping introduces a bit more diversity in the subsets that each predictor is ...
chekhovana's user avatar
6 votes
1 answer
2k views

Boosting with highly correlated features

I have a conceptual question. My understanding is, that Random Forest can be applied even when features are (highly) correlated. This is because with bagging, the influence of few highly correlated ...
Peter's user avatar
  • 7,526