Unit 06.02: Absolute and relative, always both

Three percentage points can be a rounding error or a transformation of the business. It depends entirely on where you started.

Absolute and relative, always both

prop.test() compares two proportions and gives an interval for their absolute difference, in percentage points. That is the figure that determines how many extra events actually occur.

The relative difference — the percentage change — is what gets reported in headlines, and the same absolute difference produces wildly different relative figures depending on the base rate.

Neither is dishonest. Quoting only one of them usually is, because the relative figure sounds impressive from a small base and the absolute figure sounds negligible from a large one.

This block compares two conversion rates, then varies the base.

# Two variants, binary outcome.
converted <- c(A = 84, B = 108)
visitors  <- c(A = 800, B = 800)

test <- prop.test(converted, visitors)
cat("Rate A:", converted["A"] / visitors["A"],
    "  Rate B:", converted["B"] / visitors["B"], "\n")
cat("Difference (B - A):",
    round(converted["B"]/visitors["B"] - converted["A"]/visitors["A"], 4), "\n")
cat("95% CI for the difference: [", round(-test$conf.int[2], 4), ",",
    round(-test$conf.int[1], 4), "]\n")
cat("p =", signif(test$p.value, 3), "\n\n")

# The same absolute difference means very different things at different bases.
cat("3 percentage points from different starting rates:\n")
for (base in c(0.05, 0.30, 0.80)) {
  cat(sprintf("  %.0f%% -> %.0f%%  is a relative change of %+.1f%%\n",
              base * 100, (base + 0.03) * 100, 0.03 / base * 100))
}
cat("\nReport both: the absolute difference decides workload, the relative\n")
cat("one is what people quote.\n")

Rates of 0.105 and 0.135 give an absolute difference of 3 percentage points, with an interval from −0.31 to 6.31 points and p = 0.0768 — so this particular comparison is not conclusive. The base-rate table is the lesson: the same 3 points is a +60% relative change from a 5% base, +10% from 30%, and +3.8% from 80%. One number, three completely different headlines.

The mistake this prevents

The mistake is reporting the relative change alone. 'Conversion up 60%' from a 5% base is three extra customers per hundred, and the sentence does not say so.

Takeaway

Report the absolute difference with its interval and the relative change alongside it. When the base rate is small, expect the relative figure to be the one people repeat.

Mark complete