Unit 06.01: Subtract the person out

When each unit gives you two measurements, ttest_ind throws away most of your evidence.

Subtract the person out

In a paired design — before and after, left and right, matched pairs — the same unit contributes both values. The relevant quantity is the within-unit difference, and analysing those differences removes the between-unit variation entirely.

That matters when people differ from each other much more than the treatment changes any one of them, which is the usual situation. The between-person spread is noise for this question, and pairing deletes it.

stats.ttest_rel(after, before) is a one-sample test on the differences. The gain in precision is exactly the between-unit variation you removed.

This block analyses the same thirty before-and-after pairs both ways.

import numpy as np
from scipy import stats

rng = np.random.default_rng(302)
before = rng.normal(62, 12, 30)
after = before + rng.normal(2.5, 3, 30)     # small, consistent gain

print(f"Between-person SD      : {before.std(ddof=1):.2f}")
print(f"Within-person change SD: {(after - before).std(ddof=1):.2f}\n")

ind = stats.ttest_ind(after, before, equal_var=False)
rel = stats.ttest_rel(after, before)
ci_i, ci_r = ind.confidence_interval(), rel.confidence_interval()

print(f"Independent: p = {ind.pvalue:.4f}   CI [{ci_i.low:.2f}, {ci_i.high:.2f}]"
      f"   width {ci_i.high - ci_i.low:.2f}")
print(f"Paired     : p = {rel.pvalue:.6f}   CI [{ci_r.low:.2f}, {ci_r.high:.2f}]"
      f"   width {ci_r.high - ci_r.low:.2f}")
print()
print(f"The paired interval is {(ci_i.high - ci_i.low) / (ci_r.high - ci_r.low):.1f}"
      " times narrower.")
print("Pairing works when between-unit variation is large relative to the")
print(f"effect -- here {before.std(ddof=1) / (after - before).std(ddof=1):.1f}"
      " times larger -- because it subtracts that variation out.")

The between-person SD is 14.07 and the within-person change SD only 2.80 — people differ from each other about 5.0 times more than the treatment moves any of them. Analysed as independent groups, p = 0.7062 with an interval of width 14.47: nothing. Paired, p = 0.011970 with an interval from 0.32 to 2.41, 6.9 times narrower. Same numbers, and only one analysis can see the effect.

The mistake this prevents

The mistake is running an unpaired test on paired data. It is conservative rather than wrong, which is why it goes unnoticed — you simply fail to find things that are there.

Takeaway

Whenever two measurements come from the same unit, use ttest_rel. Report both the between-unit SD and the within-unit change SD so the gain from pairing is visible.

Mark complete