1
$\begingroup$

About the methodology to find confidence and/or prediction intervals in, let's say, a regression problem, I know 2 main options:

  1. Checking normality in the estimates/predictions distribution, and applying well known Gaussian alike methods to find those intervals if the distribution is gaussian
  2. Applying non-parametric methodologies like bootstraping, so we do not need to assume/check/care whether our distribution is normal

With this in mind, I would basically always go for the second one because:

  • it is meant to be generic, as it does not assume any kind of distribution
  • it feels more like experimental as you can freely run as many iterations as you want (well, if it is computionally feasible)

The only drawback I could see is the computational cost, but it could be parallelized...

Can anyone give another point/advice?

$\endgroup$
-1
$\begingroup$

The simplest type of bootstrapped confidence intervals, percentile intervals (described in simple terms here) are by no means the only way of creating bootstrapped CIs. You should think of bootstrapped confidence intervals as a family of techniques - there are some examples listed here along with some good information around the assumptions underlying the use of bootstrapped statistics.

How many original samples you need in order for bootstrapped confidence intervals to converge varies. Like the parametric approach, this convergence will arrive at the wrong value to the extent that your sample population is not representative of the global population you are estimating. A rule of thumb is to use parametric approaches if the sample population is n < 30, but in principle bootstrap could sometimes work with smaller samples (see this thread). If you have 30 samples or more, standard practice is to draw >1000 bootstrap samples, but the important thing is that you see the estimates converging.

If you want to use bootstrap CIs you should be really explicit about the approach you have used whenever you present your analysis, because different ways of producing them can lead to different intervals.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for your answer, but I did not ask how to implement that which I already know, but comparing parametric VS non-parametric further... $\endgroup$ – German C M Apr 11 at 18:56
  • $\begingroup$ OK, sorry! What in particular do you want to know? $\endgroup$ – Nicholas James Bailey Apr 11 at 19:22
  • $\begingroup$ Maybe this edit is more helpful? $\endgroup$ – Nicholas James Bailey Apr 11 at 23:48
  • $\begingroup$ Thanks for your edit Nicholas, nevertheless you misunderstood what I said: I actually know what bootstrapping is and how to apply to CIs and PIs. What I propose in my question is to wonder when/why it should be better to try a parametric method (i.e. with gaussian distribution properties) given that non-parametric is applicable to any kind of distribution (which I know, I am not asking what bootstrap is). $\endgroup$ – German C M Apr 12 at 0:09
  • $\begingroup$ You could edit your question so that it is clear you mainly want to know when to use parametric confidence intervals - the way it currently reads is “Has anyone got any thoughts about what considerations are required before using bootstrapped confidence intervals”. I have added some information to my answer above. $\endgroup$ – Nicholas James Bailey Apr 12 at 8:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.