Why use bootstrapping?

Question

The wiki page for bootstrapping says that you use it in the case where the underlying distribution is unknown. Why is bootstrapping, or sampling with replacement, better than just calculating the variance and other properties from the data directly?

Answer 1

In general when you are evaluating statistics there are two main approaches: either you use some distributional assumptions (a generating model for your data) or not. Whenever you can you would probably go with a statistical model. But often the distributional assumptions can be wrong or they are not available to you.

If you are doing inferences about the mean of the population, than you can rely on central limit theorem and build a statistic for that. The same for variance. However what about estimating something which does not rely on general assumptions? Bootstrap works for any kind of statistic, this is where it lies its power. It's simple, and does require only minimal assumptions.

And there is another problem which appears in practice. Estimating mean rely on central limit theorem. It is true that your set of assumptions required only independent and identical distributed data. But it's something more. CLT works on large samples, it works in the limit. For small samples it can go wrong more often. And what about if your distribution is skewed. While it's true that CLT relies only on i.i.d. the needed sample size for estimating a mean on a skewed distribution is in general greater that for a symmetric distribution.

I found bootstrapping very useful in two main situations: when the sample is fairly small (but not tiny) and when the distribution is not clean (suppose it's a mixture of two distributions). And is very useful because you can build not only estimates of the population parameters, but also estimations for confidence intervals of p-values. And because it's easy to apply I also found sometimes useful even when you are assuming some model. Doing in parallel can help you to see if there is something really spooky in your sample or assumptions if inferences produces too different results, it makes you think or re-evaluate your setup.

Answer 2

In many real-world problems (especially in cases where data are the result of measurements) the amount of data available is relatively small and it is very costful to get more measurements. Luckily, modern data science is based on computers, where we can repeat simulations of our experiment to produce more data. Bootstrapping actually creates bigger samples out of your existing data and can even do this in multiple repetitions to allow significance testing.

Bootstrapping respects the distribution that your existing data have, so you don't need to make any assumptions regarding the distribution of your data. This is important because in most statistical approaches there are such assumptions, which sometimes we don't even realise or check before we apply them. (e.g. have a look here or at this post)

Bootstrapping is commonly used for the calculation of confidence intervals or for hypothesis testing. If you are using python, you might find the following links useful:

-Calculation of confidence intervals with bootstrapping example

-2-paired hypothesis testing with bootstrapping