Unit 04.03: Resample when there is no formula

When there is no formula for your statistic's uncertainty, resample the data and watch the statistic move.

Let the sample stand in for the population

The bootstrap treats your sample as the best available picture of the population. Draw from it with replacement, same size, thousands of times; compute the statistic on each resample; the spread of those values estimates the sampling distribution.

Its appeal is generality. Formulas exist for the mean and the proportion; for a median, a ratio, a trimmed mean or a difference of correlations they are awkward or absent. The bootstrap needs none.

It is not magic. It cannot repair a biased sample, and with very small n there is too little information to resample from.

This block bootstraps both a mean and a median from skewed data.

import numpy as np
from scipy import stats

rng = np.random.default_rng(104)
observed = np.round(rng.lognormal(3.4, 0.8, 40), 2)
print(f"n = {observed.size}   mean = {observed.mean():.2f}"
      f"   median = {np.median(observed):.2f}\n")

boot_means = np.array([rng.choice(observed, observed.size, replace=True).mean()
                       for _ in range(5000)])
lo, hi = np.percentile(boot_means, [2.5, 97.5])
print("Bootstrap distribution of the mean:")
print(f"  centre : {boot_means.mean():.2f}")
print(f"  SE     : {boot_means.std(ddof=1):.2f}")
print(f"  95% percentile interval: {lo:.2f} to {hi:.2f}")

t_ci = stats.ttest_1samp(observed, popmean=0).confidence_interval()
print(f"\nt-interval for comparison: {t_ci.low:.2f} to {t_ci.high:.2f}")

boot_med = np.array([np.median(rng.choice(observed, observed.size, replace=True))
                     for _ in range(5000)])
mlo, mhi = np.percentile(boot_med, [2.5, 97.5])
print(f"\n95% interval for the MEDIAN: {mlo:.2f} to {mhi:.2f}")
print("There is no standard textbook formula for that one.")
print("\nResample WITH replacement at the original size. Without replacement")
print("you only reorder the same values and every resample has one mean.")

From 40 skewed observations with a mean of 49.22, 5000 resamples give a bootstrap centre of 49.10, a standard error of 5.77, and a 95% percentile interval from 38.72 to 61.12. The t-interval on the same data runs 37.58 to 60.85 — close, which is reassurance that both are working. The median's interval, 28.52 to 53.12, has no standard textbook formula at all.

The mistake this prevents

The mistake is resampling with replace=False, which just reorders the same 40 values so every resample has an identical mean. The bootstrap then reports zero uncertainty.

Takeaway

Use the bootstrap when your statistic has no simple formula. Resample with replacement at the original sample size, use at least a few thousand resamples, and seed the generator.

Mark complete