Unit 04.00: The estimate moves even when nothing else does

Every sample from the same population gives a different answer. Inference exists because of that, and numpy makes it visible in five lines.

The estimate moves even when nothing else does

In real work you have one sample and cannot see the population. In a simulation you can build the population, so you know the true answer and can watch how far individual samples land from it.

What that shows is that variation between samples is not error in any ordinary sense. Nobody made a mistake. Drawing 30 items from a varied population simply gives a different mean each time, and the spread of those means is a measurable quantity.

That spread is the whole subject. A standard error estimates it, a confidence interval expresses it, and a p-value asks how surprising an observed value is relative to it.

This block builds a population, draws five samples, then a thousand.

import numpy as np

rng = np.random.default_rng(101)
population = rng.normal(500, 100, 100_000)
print(f"True population mean: {population.mean():.2f}\n")

for i in range(1, 6):
    s = rng.choice(population, 30, replace=False)
    print(f"Sample {i} of 30: mean {s.mean():7.2f}"
          f"  (off by {s.mean() - population.mean():+6.2f})")

many = np.array([rng.choice(population, 30, replace=False).mean()
                 for _ in range(1000)])
print(f"\nAcross 1000 samples of 30:")
print(f"  mean of the sample means : {many.mean():.2f}")
print(f"  spread of the means (SD) : {many.std(ddof=1):.2f}")
print(f"  range                    : {many.min():.1f} to {many.max():.1f}")
print("\nNo single sample was wrong. They vary because sampling varies, and")
print("the spread of that variation is what the rest of this course measures.")

The population mean is 500.19. Five samples of 30 give means from 484.05 to 524.42 — one is 24.23 above the truth and another 16.14 below. Across 1000 samples the mean of the sample means is 500.27, very close to the truth, while their spread is 18.22 and the extremes run from 434.5 to 576.6. The procedure is unbiased and any single sample can still be a long way out.

The mistake this prevents

The mistake is treating one sample's mean as the population's. It is the best estimate available and it is not the answer, and the gap between those two statements is what this module quantifies.

Takeaway

When a sampling idea is unclear, simulate it: build a population, draw repeatedly, and look at the spread of the estimates. It settles arguments that theory alone leaves ambiguous.

Mark complete