Unit 04.00: The estimate moves even when nothing else does

Every sample from the same population gives a different answer. Inference exists because of that fact, and simulation is the fastest way to see it.

The estimate moves even when nothing else does

In real work you have one sample and cannot see the population. In a simulation you can create the population, so you know the true answer and can watch how far individual samples land from it.

What that shows is that variation between samples is not error in any ordinary sense. Nobody made a mistake. Drawing 30 items from a varied population simply gives a different mean each time, and the spread of those means is a measurable quantity.

That spread is the whole subject. A standard error estimates it, a confidence interval expresses it, and a p-value asks how surprising an observed value is relative to it.

This block builds a population, draws five samples, then a thousand.

# The population is fixed. Every sample from it is different. That is the
# whole reason inference exists, and simulation is how you see it.
set.seed(101)
population <- rnorm(100000, mean = 500, sd = 100)
cat("True population mean:", round(mean(population), 2), "\n\n")

for (i in 1:5) {
  s <- sample(population, 30)
  cat(sprintf("Sample %d of 30: mean %.2f  (off by %+.2f)\n",
              i, mean(s), mean(s) - mean(population)))
}

many <- replicate(1000, mean(sample(population, 30)))
cat("\nAcross 1000 samples of 30:\n")
cat("  mean of the sample means:", round(mean(many), 2), "\n")
cat("  spread of the sample means (SD):", round(sd(many), 2), "\n")
cat("  range:", round(min(many), 1), "to", round(max(many), 1), "\n\n")
cat("No single sample was wrong. They vary because sampling varies.\n")

The population mean is 500.29. Five samples of 30 give means from 471.84 to 536.57 — one is 36.28 above the truth and another 28.45 below. Across 1000 samples the mean of the sample means is 499.17, very close to the truth, while their spread is 18.01 and the extremes run from 429.9 to 552.8. The procedure is unbiased and any single sample can still be a long way out.

The mistake this prevents

The mistake is treating one sample's mean as the population's. It is the best estimate available and it is not the answer, and the gap between those two statements is what the rest of this course quantifies.

Takeaway

When a sampling idea is unclear, simulate it: build a population, draw repeatedly, and look at the spread of the estimates. It settles arguments that theory alone leaves ambiguous.

Mark complete