Unit 04.04: Wilson intervals, and width against n

An interval says what range of values the data is consistent with. For proportions, the textbook formula is the wrong default.

The interval is the result

For a mean, scipy.stats.ttest_1samp(...).confidence_interval() gives the interval directly. For a proportion, statsmodels.stats.proportion.proportion_confint offers several methods, and Wilson is the one to use — the textbook normal approximation misbehaves badly at small n and at proportions near 0 or 1, sometimes producing bounds outside [0, 1].

Width is driven by sample size through the same square root as the standard error, so quadrupling n roughly halves the width. That is the practical fact behind every sample-size argument you will have.

Reporting the interval instead of the point estimate changes the conversation.

This block builds both kinds of interval, then varies n.

import numpy as np
from scipy import stats
from statsmodels.stats.proportion import proportion_confint

rng = np.random.default_rng(105)
values = rng.normal(72, 11, 50)
res = stats.ttest_1samp(values, popmean=0)
ci = res.confidence_interval()
print(f"Mean   : {values.mean():.2f}")
print(f"95% CI : {ci.low:.2f} to {ci.high:.2f}   width {ci.high - ci.low:.2f}\n")

successes, n = 84, 200
lo, hi = proportion_confint(successes, n, method="wilson")
print(f"Proportion : {successes / n}")
print(f"95% CI     : {lo:.4f} to {hi:.4f}   width {hi - lo:.4f}\n")

print("Width against sample size, same proportion:")
for nn in (50, 200, 800, 3200):
    lo, hi = proportion_confint(round(0.42 * nn), nn, method="wilson")
    print(f"  n = {nn:4d}   CI {lo:.3f} to {hi:.3f}   width {hi - lo:.3f}")
print("\nEach fourfold increase in n roughly halves the width. Wilson is the")
print("default worth using -- the textbook normal interval misbehaves badly")
print("for small n or proportions near 0 and 1.")

The mean of 71.57 carries an interval from 68.41 to 74.73, width 6.32. The proportion of 0.42 from 200 observations has a Wilson interval from 0.354 to 0.489, width 0.1355 — thirteen and a half percentage points, far more uncertainty than '42%' suggests. The sample-size table shows the width falling from 0.264 at n = 50 to 0.034 at n = 3200: each fourfold increase roughly halves it.

The mistake this prevents

The mistake is reporting the point estimate alone. '42% completed' sounds precise; from 200 observations the data is consistent with anything from 35% to 49%, which may straddle the threshold the decision turns on.

Takeaway

Report an interval with every estimate and read its width before its centre. Use Wilson intervals for proportions, and remember the square-root rule when someone asks for more precision.

Mark complete