Questions tagged [bootstraping]
The bootstraping tag has no usage guidance.
20 questions
4
votes
1
answer
50
views
Nested-cross validation pipeline and confidence intervals
I'm hoping someone can help me think through this. I've come across a lot of different resources on nested-cv, but I think I'm confused as to how to go about model selection and the appropriate ...
0
votes
0
answers
9
views
Is this the correct way to model a sample statistic: Bayesian vs Bootstrapping
I have 2 samples S1 & S2. Contain lead level margins, converted 1 or 0 flag. Need to compare metric sum(margin) /sum(converted) for both. Sample size is low for frequentist. I tried bootstrapping ...
- 1
0
votes
0
answers
18
views
Is there a way to create a bootstrapped beta calibration function to use on new data?
I have created ML classification models that are now to be evaluated on a different population for external validation (n=5000, event rates between n=400 and n=1200 for different outcomes under study)....
- 1
0
votes
0
answers
33
views
Estimate confidence interval of a cubic B-spline fit
I have a non-linear system. I apply multiple inputs x(t) (t=time) and measure each response y(t).
In other words, I have inputs x1(t), x2(t), x3(t), xn(t) and I measure y1(t), y2(t), y3(t), yn(t) for ...
- 1
1
vote
1
answer
16
views
Model evaluation approach allowing manual experimentation without data leakage
In supervised machine learning, are there any evaluation approaches beside using a fixed holdout test dataset, which allow me as a scientist to manually compare preprocessing approaches, without ...
- 111
0
votes
0
answers
92
views
Understanding the code for calculating the 95% confidence interval of AUC using bootstrapping
It's really embarrassing, but I lack statistical knowledge. I would like to find the confidence interval for AUC at 95%. Actually, I got the code from here(https://stackoverflow.com/questions/52373318/...
- 13
3
votes
2
answers
302
views
How bootstrapping works for prediction intervals?
I'm experimenting with prediction interval (PI) over univariant time-data using skforecast pythonic package..
in the documentation it is mentioned that:
Prediction intervals
A prediction interval ...
- 432
1
vote
0
answers
244
views
What is the best way to combine cross-validation and bootstrapping for one application?
We intend to model data with non-parametric covariate splines and we would like to understand the uncertainty of the parameter estimates/response estimates.
Currently, we use cross-validation to model ...
- 111
0
votes
1
answer
212
views
Understanding bootstrapping in bias variance decomposition
I was going through bias and variance tradeoff article and it makes use of bias_variance_decomp function from mlxtend library. ...
- 219
1
vote
0
answers
83
views
How to perform bootstrap validation on CART decision tree?
I have a relatively small dataset n = 500 for which I am training a CART decision tree.
My dataset has about 30 variables and the outcome has 3 classes.
I am using CART for interpretability purposes, ...
- 11
1
vote
0
answers
64
views
Evaluate Dendrogram Statistical Significance
I have N=21 objects and each one has about 80 possible not NaN descriptors.
I carried out a hierarchical clustering on the objects and I obtained this dendrogram.
I want some kind of 'confidence' ...
- 111
1
vote
0
answers
35
views
Stratified sampling - use of proxy variable
For splitting of the data into train/test/val I use stratified sampling. Is it appropriate to define strata using information extracted from the dataset? E.g. use machine-learning to model proxy ...
- 11
4
votes
3
answers
675
views
List of samples that each tree in a random forest is trained on in Scikit-Learn
In Scikit-learn's random forest, you can set bootstrap=True and each tree would select a subset of samples to train on. Is there a way to see which samples are used in each tree?
I went through the ...
1
vote
0
answers
15
views
How are the same observation sets treated in Random Forests with Bootstrapping?
Let's assume an extremely small dataset with only 4 observations. And I create a Random Forest model, with a quite large number of trees, say 200. If so, some sample sets that are the same each other ...
- 11
1
vote
1
answer
2k
views
nnet in caret. Bootstrapping or cross-validation?
I want to train shallow neural network with one hidden layer using nnet in caret. In trainControl, I used method = "cv" to perform 3-fold cross-validation. The snipped the code and results ...
- 125
3
votes
0
answers
151
views
Difference Bagging and Bootstrap aggregating
Bootstrap belongs to Efron. Tibshirani wrote a book about that in reference to Efron.
Bootstrap process for estimating the standard error of statistic s(x). B bootstrap sample are generatied from ...
- 329
2
votes
2
answers
287
views
Resampling train and test data in R
I need to try out few different machine learning methods (SVM, Logistic regression etc.), predict a value either true or false, and write down their AUC and Accuracy of these predictions.
I have ...
- 21
2
votes
1
answer
65
views
Question on bootstrap sampling
I have a corpus of manually annotated (aka "gold standard) documents and a collection of NLP systems annotations on the text from the corpus. I want to do a bootstrap sampling of the system and ...
- 201
0
votes
1
answer
185
views
Perform bootstrapping of an ordinary linear regression model, using B=100 bootstrap resamples of my dataset, and getting RMSE
So Im studying machine learning through R, and Im working with the boston data set from the library MASS. I am practicing bootsrapping. I already carried out analysis to determine how ,many distinct ...
1
vote
1
answer
121
views
About confidence/prediction intervals: parametric methods VS non-parametric (via bootstrap) methods
About the methodology to find confidence and/or prediction intervals in, let's say, a regression problem, I know 2 main options:
Checking normality in the estimates/predictions distribution, and ...
- 2,734