Unit 04.02: Two different spreads, and ddof again

The standard deviation describes the spread of values. The standard error describes the spread of your estimate. Confusing them is the commonest error in applied statistics.

Two different spreads, and ddof again

The standard deviation is a property of the data: how far individual values sit from their mean. It does not shrink as you collect more data, because the population's variability is what it is.

The standard error is a property of an estimate: how far a sample mean would sit from the truth across repeated samples. It falls as n grows.

SE = SD / sqrt(n) connects them, and that square root is why precision is expensive. It also matters that you use the *sample* standard deviation: numpy's default ddof=0 understates it.

This block computes both from one sample and checks the formula against simulation.

import numpy as np

rng = np.random.default_rng(103)
population = rng.normal(500, 100, 200_000)
sample = rng.choice(population, 30, replace=False)

print("Standard deviation describes the SPREAD OF VALUES:")
print(f"  sd of this sample of 30 : {sample.std(ddof=1):.2f}")
print()
print("Standard error describes the SPREAD OF THE ESTIMATE:")
se_formula = sample.std(ddof=1) / np.sqrt(sample.size)
print(f"  SE = sd / sqrt(n)       : {se_formula:.2f}")
print()

simulated = np.array([rng.choice(population, 30, replace=False).mean()
                      for _ in range(4000)]).std(ddof=1)
print(f"Simulated SD of 4000 sample means : {simulated:.2f}")
print(f"Formula estimate from ONE sample  : {se_formula:.2f}")
print(f"  they differ by {abs(se_formula - simulated) / simulated * 100:.0f}%,"
      " because the one sample's sd is itself an estimate")
print(f"  (this sample's sd {sample.std(ddof=1):.1f} against the population's"
      f" {population.std(ddof=1):.1f})")
print()
print("Note ddof. numpy's default is 0, the population formula; for a sample")
print("standard deviation you want ddof=1:")
print(f"  sample.std()        = {sample.std():.3f}")
print(f"  sample.std(ddof=1)  = {sample.std(ddof=1):.3f}")
print()
print("Quadrupling n halves the standard error:")
for n in (30, 120, 480):
    print(f"  n = {n:3d}   SE = {population.std(ddof=1) / np.sqrt(n):.2f}")
print(f"  halving it once more needs n = {480 * 4}")

The sample of 30 has a standard deviation of 112.82, giving a standard error of 20.60. Simulating 4000 sample means directly gives 18.21 — they differ by 13%, and the block says why: this sample's standard deviation is 112.8 against the population's 100.0, so the one-sample estimate of the standard error is itself uncertain. The ddof difference is visible too: 110.919 against 112.815. The cost of precision is in the last lines — SE falls from 18.26 at n = 30 to 4.57 at n = 480, and halving it once more needs n = 1920.

The mistake this prevents

The mistake is quoting the standard error as though it described the spread of the data. It makes the data look far more consistent than it is, and the error grows with sample size.

Takeaway

Use the standard deviation to describe data and the standard error to describe an estimate's precision, always with ddof=1. Label which one a figure or an error bar shows.

Mark complete