Unit 05.00: A claim precise enough to be tested

The null hypothesis is not what you believe. It is a specific numerical claim you set up in order to see whether the data can knock it down.

A claim precise enough to be tested

H0 has to be specific — the mean is 100, the two proportions are equal — because the whole procedure works by computing what data would look like if it were true. A vague claim gives nothing to compute against.

H1 is what you conclude if H0 falls. It is stated in advance, and whether it is one-sided or two-sided is part of the statement.

The asymmetry matters. Rejecting H0 is a positive finding. Failing to reject it is not evidence that H0 is true — it means the data was not sufficient to rule it out, which is a very different thing and depends heavily on the sample size.

This block tests two samples, one drawn from a world where H0 holds and one where it does not.

# The null is a specific claim about the population, not a belief about it.
set.seed(201)

# H0: the mean is 100.  H1: the mean is not 100.  Two-sided.
sample_a <- rnorm(40, mean = 100, sd = 15)   # H0 is TRUE here
sample_b <- rnorm(40, mean = 108, sd = 15)   # H0 is FALSE here

for (nm in c("sample_a", "sample_b")) {
  s <- get(nm)
  tt <- t.test(s, mu = 100)
  cat(sprintf("%s: mean %.2f  p = %.4f  -> %s\n", nm, mean(s), tt$p.value,
              ifelse(tt$p.value < 0.05, "reject H0", "do not reject H0")))
}

cat("\nNote what sample_a's outcome does NOT say.\n")
cat("'Do not reject' is not 'H0 is true'. With n = 40 and sd = 15 the test\n")
cat("simply cannot distinguish a mean of 100 from a mean of 103.\n")
cat("The alternative must be stated before the data: two-sided here, so a\n")
cat("difference in either direction counts as evidence.\n")

The first sample has a mean of 97.93 and gives p = 0.3229, so the test does not reject. That is correct — H0 really is true for it — but the same outcome would appear if the true mean were 103, because at n = 40 with an SD of 15 the test cannot tell those apart. The second sample, mean 107.42, gives p = 0.0063 and rejects.

The mistake this prevents

The mistake is reporting 'no significant difference' as 'no difference'. The first result above is compatible with H0 being exactly true and with it being wrong by several points.

Takeaway

State H0 as a specific number and H1, including its sidedness, before looking at the data. Never write 'we found no difference' when what happened is that you failed to reject.

Mark complete