Unit 06.00: One sample against a target, two against each other

Two questions, two tests: is this mean different from a target, and are these two groups different from each other?

One sample against a target, two samples against each other

A one-sample test compares a mean to a fixed value that came from outside the data — a specification, a target, a previous period. A two-sample test compares two independent groups measured in the same study.

R's default for two samples is Welch's t-test, which does not assume the two groups have equal variances. The classical equal-variance test is available and is almost never worth choosing: when the variances really are equal the two agree closely, and when they are not — especially with unequal group sizes — the equal-variance version gives the wrong answer with confident-looking degrees of freedom.

This block runs a one-sample test, then a two-sample test on groups with deliberately unequal spreads and sizes.

set.seed(301)

# One-sample: is this mean different from a fixed target?
weights <- rnorm(40, mean = 502, sd = 5)
one <- t.test(weights, mu = 500)
cat("One-sample against a 500g target\n")
cat("  mean", round(mean(weights), 2), " p =", signif(one$p.value, 3),
    " CI [", round(one$conf.int[1], 2), ",", round(one$conf.int[2], 2), "]\n\n")

# Two-sample: do two independent groups differ?
line_a <- rnorm(60, 502, 4)      # tight, larger sample
line_b <- rnorm(15, 496, 14)     # variable, smaller sample
two <- t.test(line_a, line_b)
cat("Two-sample, line A against line B\n")
cat("  means", round(mean(line_a), 2), "and", round(mean(line_b), 2), "\n")
cat("  difference", round(as.numeric(diff(rev(two$estimate))), 2),
    " p =", signif(two$p.value, 3),
    " CI [", round(two$conf.int[1], 2), ",", round(two$conf.int[2], 2), "]\n\n")

cat("SDs:", round(sd(line_a), 2), "and", round(sd(line_b), 2),
    " ns:", length(line_a), "and", length(line_b), "\n")
cat("Equal-variance p:", signif(t.test(line_a, line_b, var.equal = TRUE)$p.value, 3),
    " df", round(t.test(line_a, line_b, var.equal = TRUE)$parameter, 1), "\n")
cat("Welch p         :", signif(two$p.value, 3),
    " df", round(two$parameter, 1), "\n")
cat("\nWith unequal spreads AND unequal group sizes the two disagree, and the\n")
cat("equal-variance version is the one that is wrong. Welch is R's default.\n")

The one-sample test puts the mean at 501.84 against a 500g target, p = 0.0114, interval 500.44 to 503.24 — a real but small overshoot. The two-sample comparison has SDs of 3.82 and 13.46 from groups of 60 and 15, and here the two methods diverge: equal-variance gives p = 0.575 on 73 degrees of freedom, Welch gives 0.756 on 14.6. Welch's much smaller df reflects that the noisy group of 15 carries most of the uncertainty, and the equal-variance figure is the wrong one.

The mistake this prevents

The mistake is running the equal-variance test because it is the one in the textbook. With unequal spreads and unequal group sizes it systematically misstates the uncertainty.

Takeaway

Use t.test() as it comes — Welch is the default for a good reason. Report both group SDs and both sample sizes so a reader can see whether the assumption ever mattered.

Mark complete