Unit 02.03: Repeated measurements are not independent evidence

Three measurements from one person are not three independent pieces of evidence, and treating them as such makes your interval narrower than the data has earned.

Independence is an assumption you can check

Nearly every standard test assumes the observations are independent. Repeated measurements break that assumption: two readings from the same person are more alike than two readings from different people, so they carry less information than their count suggests.

The consequence is one-directional and therefore dangerous. Counting dependent rows as independent inflates n, shrinks the standard error, narrows the interval and lowers the p-value. It always makes results look stronger, never weaker.

The simplest correct approach at this level is to aggregate to one row per unit first. Methods that use all the rows properly — mixed models — exist and are a later topic.

This block analyses the same readings both ways.

suppressPackageStartupMessages(library(dplyr))

visits <- data.frame(
  user_id = c(1, 1, 1, 2, 3, 3, 4, 5),
  score   = c(70, 74, 72, 88, 61, 65, 79, 83)
)

cat("Rows:", nrow(visits), " Distinct users:", n_distinct(visits$user_id), "\n\n")

naive <- t.test(visits$score, mu = 70)
cat("Treating every row as independent: n =", length(visits$score),
    ", CI", round(naive$conf.int[1], 1), "to", round(naive$conf.int[2], 1), "\n")

per_user <- visits |> group_by(user_id) |> summarise(score = mean(score), .groups = "drop")
correct <- t.test(per_user$score, mu = 70)
cat("One row per user            : n =", nrow(per_user),
    ", CI", round(correct$conf.int[1], 1), "to", round(correct$conf.int[2], 1), "\n\n")

cat("Interval width, row-level :", round(diff(naive$conf.int), 2), "\n")
cat("Interval width, user-level:", round(diff(correct$conf.int), 2), "\n")
cat("\nRepeated measurements from one person are not independent evidence.\n")
cat("Counting them as such makes the interval narrower than the data earns.\n")

Eight rows come from 5 users. Treating every row as independent gives n = 8 and an interval from 66.4 to 81.6, width 15.11. Aggregating to one row per user gives n = 5 and an interval from 64.9 to 89.1, width 24.27 — sixty percent wider. Both used every observation. Only the second respects where the observations came from.

The mistake this prevents

The mistake is checking for exactly duplicated rows and concluding the data is independent. Repeated measurements are not duplicates; they are different values from the same unit, and no duplicate check will find them.

Takeaway

Identify the unit and count distinct units alongside rows. Aggregate to one row per unit before testing, and report the number of units, not just the number of rows.

Mark complete