Unit 06.02: Absolute and relative, always both

Three percentage points can be a rounding error or a transformation of the business. It depends entirely on where you started.

Absolute and relative, always both

proportions_ztest compares two proportions and confint_proportions_2indep gives the interval for their absolute difference in percentage points. That figure determines how many extra events actually occur.

The relative difference — the percentage change — is what gets reported in headlines, and the same absolute difference produces wildly different relative figures depending on the base rate.

Neither is dishonest. Quoting only one of them usually is, because the relative figure sounds impressive from a small base and the absolute figure sounds negligible from a large one.

This block compares two conversion rates, then varies the base.

import numpy as np
from statsmodels.stats.proportion import proportions_ztest, confint_proportions_2indep

converted = np.array([84, 108])
visitors = np.array([800, 800])
rates = converted / visitors

stat, p = proportions_ztest(converted, visitors)
lo, hi = confint_proportions_2indep(converted[1], visitors[1],
                                    converted[0], visitors[0],
                                    compare="diff", method="wald")

print(f"Rate A: {rates[0]:.4f}   Rate B: {rates[1]:.4f}")
print(f"Difference (B - A): {rates[1] - rates[0]:+.4f}")
print(f"95% CI for the difference: [{lo:+.4f}, {hi:+.4f}]")
print(f"p = {p:.4f}\n")

print("Three percentage points from different starting rates:")
for base in (0.05, 0.30, 0.80):
    print(f"  {base:.0%} -> {base + 0.03:.0%}  is a relative change of"
          f" {0.03 / base:+.1%}")
print()
print("Report both. The absolute difference decides workload; the relative")
print("one is what gets quoted, and it depends entirely on the base rate.")

Rates of 0.1050 and 0.1350 give an absolute difference of +3.00 percentage points, with an interval from −0.18 to +6.18 points and p = 0.0648 — so this particular comparison is not conclusive. The base-rate table is the lesson: the same 3 points is +60.0% from a 5% base, +10.0% from 30%, and +3.8% from 80%. One number, three completely different headlines.

The mistake this prevents

The mistake is reporting the relative change alone. 'Conversion up 60%' from a 5% base is three extra customers per hundred, and the sentence does not say so.

Takeaway

Report the absolute difference with its interval and the relative change alongside it. When the base rate is small, expect the relative figure to be the one people repeat.

Mark complete