Unit 04.03: Resample when there is no formula

When there is no formula for your statistic's uncertainty, resample the data and watch the statistic move.

Let the sample stand in for the population

The bootstrap treats your sample as the best available picture of the population. Draw from it *with replacement*, same size, thousands of times; compute the statistic on each resample; the spread of those values estimates the sampling distribution.

Its appeal is generality. Formulas exist for the mean and the proportion; for a median, a ratio of medians, a trimmed mean or a correlation difference, they are awkward or absent. The bootstrap needs none.

It is not magic. It cannot repair a biased sample, and with very small n there is too little information to resample. But for a moderate sample and an unusual statistic it is the most useful tool in this course.

This block bootstraps both a mean and a median from skewed data.

set.seed(104)
# A skewed sample where the mean's sampling distribution is not obvious.
observed <- round(rlnorm(40, meanlog = 3.4, sdlog = 0.8), 2)
cat("n =", length(observed), " mean =", round(mean(observed), 2),
    " median =", round(median(observed), 2), "\n\n")

# Resample WITH replacement, same size, thousands of times.
boot_means <- replicate(5000, mean(sample(observed, length(observed), replace = TRUE)))

cat("Bootstrap distribution of the mean:\n")
cat("  centre:", round(mean(boot_means), 2), "\n")
cat("  SE    :", round(sd(boot_means), 2), "\n")
ci <- quantile(boot_means, c(0.025, 0.975))
cat("  95% percentile interval:", round(ci[1], 2), "to", round(ci[2], 2), "\n\n")

t_ci <- t.test(observed)$conf.int
cat("t-interval for comparison:", round(t_ci[1], 2), "to", round(t_ci[2], 2), "\n\n")

# The bootstrap works for statistics with no simple formula.
boot_median <- replicate(5000, median(sample(observed, length(observed), replace = TRUE)))
mci <- quantile(boot_median, c(0.025, 0.975))
cat("95% interval for the MEDIAN:", round(mci[1], 2), "to", round(mci[2], 2), "\n")
cat("There is no standard textbook formula for that one.\n")

From 40 skewed observations with a mean of 34.84, 5000 resamples give a bootstrap centre of 34.83 and a standard error of 3.54, with a 95% percentile interval from 28.15 to 42.06. The t-interval on the same data runs 27.54 to 42.13 — close, which is reassurance that both are working. The median's interval, 21.23 to 39.13, has no standard textbook formula at all.

The mistake this prevents

The mistake is resampling without replacement, which just reorders the same 40 values and gives every resample an identical mean. The bootstrap then reports zero uncertainty.

Takeaway

Use the bootstrap when your statistic has no simple formula. Resample with replacement at the original sample size, use at least a few thousand resamples, and seed the run.

Mark complete