Unit 06.01: Subtract the person out

When each unit gives you two measurements, comparing them as two independent groups throws away most of your evidence.

Subtract the person out

In a paired design — before and after, left and right, matched pairs — the same unit contributes both values. The relevant quantity is the within-unit difference, and analysing those differences removes the between-unit variation entirely.

That matters when people differ from each other much more than the treatment changes any one of them, which is the usual situation. The between-person spread is noise for this question, and pairing deletes it.

t.test(after, before, paired = TRUE) is a one-sample test on the differences. The gain in precision is exactly the between-unit variation you removed.

This block analyses the same thirty before-and-after pairs both ways.

set.seed(302)
subject <- 1:30
before  <- rnorm(30, mean = 62, sd = 12)
after   <- before + rnorm(30, mean = 2.5, sd = 3)   # small, consistent gain

cat("Between-person SD:", round(sd(before), 2), "\n")
cat("Within-person change SD:", round(sd(after - before), 2), "\n\n")

unpaired <- t.test(after, before)
paired   <- t.test(after, before, paired = TRUE)

cat("Unpaired: p =", signif(unpaired$p.value, 3),
    " CI [", round(unpaired$conf.int[1], 2), ",",
    round(unpaired$conf.int[2], 2), "]\n")
cat("Paired  : p =", signif(paired$p.value, 4),
    " CI [", round(paired$conf.int[1], 2), ",",
    round(paired$conf.int[2], 2), "]\n\n")

cat("The paired interval is", round(diff(unpaired$conf.int) /
                                    diff(paired$conf.int), 1),
    "times narrower.\n")
cat("Pairing works when between-unit variation is large relative to the\n")
cat("effect -- here", round(sd(before) / sd(after - before), 1),
    "times larger -- because it subtracts that variation out.\n")

The between-person SD is 12.77 and the within-person change SD is only 2.83 — people differ from each other about 4.5 times more than the treatment moves any of them. Analysed as independent groups, p = 0.664 with an interval from −5.27 to 8.21: nothing. Paired, p = 0.007976 with an interval from 0.42 to 2.53, 6.4 times narrower. Same numbers, and only one analysis can see the effect.

The mistake this prevents

The mistake is running an unpaired test on paired data. It is conservative rather than wrong, which is why it goes unnoticed — you simply fail to find things that are there.

Takeaway

Whenever two measurements come from the same unit, analyse the differences. Report both the between-unit SD and the within-unit change SD so the gain from pairing is visible.

Mark complete