Unit 05.00: A claim precise enough to be tested

The null hypothesis is not what you believe. It is a specific numerical claim set up so the data has something to knock down.

A claim precise enough to be tested

H0 has to be specific — the mean is 100, the two proportions are equal — because the procedure works by computing what data would look like if it were true. A vague claim gives nothing to compute against.

H1 is what you conclude if H0 falls, and whether it is one-sided or two-sided is part of the statement, fixed in advance.

The asymmetry matters. Rejecting H0 is a positive finding. Failing to reject it is not evidence that H0 is true — it means the data was not sufficient to rule it out, which depends heavily on the sample size.

This block tests two samples, one drawn from a world where H0 holds and one where it does not.

import numpy as np
from scipy import stats

rng = np.random.default_rng(201)

# H0: the mean is 100.  H1: the mean is not 100.  Two-sided, fixed in advance.
samples = {
    "null_true":  rng.normal(100, 15, 40),   # H0 really holds
    "null_false": rng.normal(108, 15, 40),   # H0 really does not
}
for name, s in samples.items():
    res = stats.ttest_1samp(s, popmean=100)
    verdict = "reject H0" if res.pvalue < 0.05 else "do not reject H0"
    print(f"{name:11s} mean {s.mean():6.2f}   p = {res.pvalue:.4f}   -> {verdict}")

print("\nNote what the first outcome does NOT say.")
print("'Do not reject' is not 'H0 is true'. At n = 40 with sd = 15 this test")
print("cannot distinguish a mean of 100 from a mean of 103.")
print("H1 must be stated before the data. Two-sided here, so a difference in")
print("either direction counts as evidence against H0.")

The first sample has a mean of 99.38 and gives p = 0.8186, so the test does not reject. That is correct — H0 really is true for it — but the same outcome would appear if the true mean were 103, because at n = 40 with an SD of 15 the test cannot tell those apart. The second sample, mean 112.53, rejects decisively.

The mistake this prevents

The mistake is reporting 'no significant difference' as 'no difference'. The first result above is compatible with H0 being exactly true and with it being wrong by several points.

Takeaway

State H0 as a specific number and H1, including its sidedness, before looking at the data. Never write 'we found no difference' when what happened is that you failed to reject.

Mark complete