Bootstrapping and Resampling

When a statistic has no clean closed-form sampling distribution, bootstrapping gives you one by resampling.

The recipe:

1. Resample $n$ values from your data with replacement. 2. Compute the statistic on that resample. 3. Repeat $B$ times — typically $B = 1{,}000$ or $10{,}000$. 4. Use the empirical distribution of those $B$ statistics as your sampling distribution.

Bootstrapping shines when computing standard errors and confidence intervals for statistics that resist analytic treatment — quantiles, Sharpe ratios, ratios of estimates, complicated transformations. It's also the default for any statistic with finite-sample bias.

Caveats: vanilla bootstrap struggles in the extreme tails (you can't sample what you didn't see) and when data isn't i.i.d. For time series, use block bootstrap or stationary bootstrap to preserve serial dependence; for hierarchical data, use a hierarchical resampling scheme.