Unit 07.06: Natural frequencies are understood most reliably

The same result stated four ways will produce four different decisions. Choosing the framing is part of the analysis, not a presentation detail.

Natural frequencies are understood most reliably

Percentage points are accurate and feel small. Relative change is persuasive and routinely misread. Natural frequencies — out of every 1000, this many before and this many after — are understood most reliably by non-specialists, and are hardest to misinterpret. Business units connect the result to a decision.

None of these is the honest one and the others dishonest. The dishonest move is picking whichever framing best supports the conclusion you already wanted, and giving only that one.

The part almost always dropped is the uncertainty, and it is the part that changes what a reader should do.

This block states one conversion result four ways, then attaches the interval.

converted <- c(control = 96, treated = 132)
n <- c(control = 1200, treated = 1200)
rates <- converted / n
abs_diff <- as.numeric(rates["treated"] - rates["control"])
test <- prop.test(converted, n)

cat("--- the same result, four ways ---\n\n")
cat("1. Percentage points:", round(abs_diff * 100, 1), "pp increase\n")
cat("2. Relative        :", round(abs_diff / rates["control"] * 100, 0),
    "% increase\n")
cat("3. Natural frequency: out of every 1000 visitors,",
    round(rates["control"] * 1000), "converted before and",
    round(rates["treated"] * 1000), "after\n")
cat("4. Business units   : at 10,000 visitors a month, about",
    round(abs_diff * 10000), "extra conversions\n\n")

cat("With the uncertainty attached, which is the part usually dropped:\n")
cat("   between", round(-test$conf.int[2] * 10000), "and",
    round(-test$conf.int[1] * 10000), "extra conversions a month\n\n")

cat("Natural frequencies are understood most reliably. Relative change is\n")
cat("the most persuasive and the most misleading -- it is the one to pair\n")
cat("with an absolute figure.\n")

The same finding is 3 percentage points, a 38% increase, a move from 80 to 110 conversions per 1000 visitors, and about 300 extra conversions a month at 10,000 visitors. With the interval attached, that last figure becomes between 57 and 543 extra conversions a month — a range wide enough that a plan built on 300 may not survive.

The mistake this prevents

The mistake is presenting only the relative change because it is the most impressive. Pairing it with an absolute figure is the minimum, and carrying the interval into the business units is what makes the number usable.

Takeaway

Give at least two framings, always including an absolute one, and prefer natural frequencies for non-technical readers. Carry the confidence interval into whatever units the decision is made in.

Mark complete