Skip to main content
4 of 4
added 1 character in body
David Marx
  • 3.3k
  • 16
  • 23

No matter the model, you can always use the non-parametric bootstrap to construct a confidence interval for any parameter, including predictions (which are actually random variables themselves but are reported as expectations). Here's the general procedure:

  1. Let $N$ denote the number of observations in your training data $X$, and $x_j$ denote the specific observation whose prediction, $\hat{y}_j$, you want a CI for.
  2. Let $K$ denote some number of resampling iterations (Must be $\ge 20$ for a CI with coverage $\ge 95\%$)
  3. For $i$ in $K$, draw a $N$ random samples from $X$ with replacement. Denote this $X_i^{*}$
  4. Train a model on $X_i^{*}$ and use this model to form a prediction on $x_j$. Call this $\hat{y}^{*}_{ji}$
  5. Estimate distributional parameters for $\hat{y}_j$ from your sample. A $100 - \alpha$ CI is given by the $\frac{\alpha}{2}$ and $100 - \frac{\alpha}{2}$ percentiles of $\hat{y}^{*}_{j}$.
David Marx
  • 3.3k
  • 16
  • 23