Unit 05.05: Read the interval, then the p-value

R's test output has five parts. Most readers look at one of them, and it is not the most useful one.

Read the interval, then the p-value

t.test() prints the statistic, the degrees of freedom, the p-value, a confidence interval and the estimates. The interval is the part that tells you how large the effect might be; the p-value only tells you whether zero is inside it.

The degrees of freedom being fractional is worth understanding rather than ignoring: R defaults to Welch's test, which does not assume the two groups have equal variances and adjusts the degrees of freedom accordingly. It is the safer default and costs almost nothing when the variances happen to match.

The confidence interval is always for the *difference*, not for either group's mean.

This block prints a complete result and then annotates each line.

set.seed(206)
control <- rnorm(35, 50, 9)
treated <- rnorm(35, 55, 9)

result <- t.test(treated, control)
print(result)

cat("\n--- what each part is ---\n")
cat("t statistic  :", round(result$statistic, 3),
    " -- difference in standard-error units\n")
cat("df           :", round(result$parameter, 2),
    " -- fractional because Welch does not assume equal variances\n")
cat("p-value      :", signif(result$p.value, 4), "\n")
cat("conf.int     :", round(result$conf.int[1], 2), "to",
    round(result$conf.int[2], 2), " -- for the DIFFERENCE\n")
cat("estimates    :", round(result$estimate[1], 2), "and",
    round(result$estimate[2], 2), " -- the two group means\n\n")
cat("The confidence interval is the most useful line and the one most often\n")
cat("skipped. Welch is R's default, and it is the safer default.\n")

t = 3.126 with 67.84 degrees of freedom — fractional, because Welch adjusted them — and p = 0.002605. The interval for the difference runs from 2.25 to 10.18, and the group means are 57.81 and 51.59. That interval is the sentence a reader needs: the treated group is somewhere between about two and ten points higher.

The mistake this prevents

The mistake is reading the confidence interval as being about one group's mean. It is the interval for the difference, and it is the only line in the output that answers 'how much'.

Takeaway

Read the interval first and the p-value second. Keep Welch's default, and quote the interval in every write-up.

Mark complete