# question about sklearn.ensemble.BaggingClassifier

I am experimenting with BagginClassifier, but I fail to get the expected functionality.

Basically, the BagginClassifier should draw (bootstrapping) a new data set with replacement. For example: the following code should generate a new bootstrapped sample of the same size as the original data set:

import sklearn.datasets as ds
import numpy as np

bag = BaggingClassifier(base_estimator=LogisticRegression(),
n_estimators=100,
max_samples=1.0,
bootstrap=True,
n_jobs=1)
bag.fit(X, y)

print X[bag.estimators_samples_[0]].shape
>> 95


(or any other number close to 95).

Naively, I would expect to get the bootstrapped sample of the same size as the original one (150), but with some random repetition of rows. However, I get a smaller sample size with unique rows. That's strange.

What's wrong here?

Found the answer hiding in lines 93-100 in the bagging.py file.
Here is what I understand - the bootstrapping process works in three steps:

1. Calculate number of samples to train each estimator on (the max_samples variable in the bagging.py code). In your case its $1.0 * x.shape[0] = 150$.

2. Select with repetition the needed max samples(as calculated in the previous step).
The selection is done by using randint function, and it generates an array of the x series indices. A given index can appear in this array more than once.

3. In order to account for indices(samples) that were selected more than one, a weights vector is passed into the base estimator fit function. So, for example, a sample that was selected twice will have $weight * 2$ and it will have the desired impact on the fitting algorithm.

As to my understanding, one can use the estimators_samples_ only to find out which samples were included and not how many times each of them did.

• thanks, that totally makes sense to me, however it is a bit frustrating, that the output of sample_ is just a collection of random indices. Why not to create a separate output, smith similar to: .estimators_indices_  to provide us with this functionality and the . .estimators_samples_ will give bootstrapped sample. – Arnold Klein Sep 27 '17 at 14:45