Unit 07.01: A difference divided by the spread

Cohen's d expresses a difference in standard deviations, which makes it comparable across measures and unreadable on its own.

A difference divided by the spread

Cohen's d is the difference between two means divided by their pooled standard deviation. It answers 'how big is this difference relative to how much the values vary', which is what allows a result on one scale to be compared with a result on another.

The price is that d responds to the spread as much as to the difference. Two studies with the same raw gap can report very different d values simply because one measured a more homogeneous group.

The conventional labels — 0.2 small, 0.5 medium, 0.8 large — are a last resort. They were offered as rough placeholders, not as a scale, and in any field where you know what the numbers mean, the raw difference is better.

This block computes d, then recomputes it against a much tighter spread.

set.seed(402)
a <- rnorm(50, 100, 15)
b <- rnorm(50, 107, 15)

pooled_sd <- sqrt(((length(a) - 1) * var(a) + (length(b) - 1) * var(b)) /
                  (length(a) + length(b) - 2))
d <- (mean(b) - mean(a)) / pooled_sd

cat("Raw difference:", round(mean(b) - mean(a), 2), "points\n")
cat("Pooled SD     :", round(pooled_sd, 2), "\n")
cat("Cohen's d     :", round(d, 3), "\n\n")

cat("d says: the groups differ by", round(d, 2), "standard deviations.\n")
cat("Rough conventions -- 0.2 small, 0.5 medium, 0.8 large -- are a last\n")
cat("resort, not a substitute for knowing your own field's scale.\n\n")

# The same raw difference is a different d when the spread differs.
tight <- rnorm(50, 107, 5)
d_tight <- (mean(tight) - mean(a)) / sqrt(((49 * var(a)) + (49 * var(tight))) / 98)
cat("Against a spread of 5 rather than 15:\n")
cat("  raw difference", round(mean(tight) - mean(a), 2),
    "-> d =", round(d_tight, 2), "\n")
cat("  raw difference", round(mean(b) - mean(a), 2),
    "-> d =", round(d, 2), "\n")
cat("  the raw gap grew by a factor of",
    round((mean(tight) - mean(a)) / (mean(b) - mean(a)), 1),
    "and d by a factor of", round(d_tight / d, 1), "\n")
cat("d responds to the spread as much as to the gap, which is what makes it\n")
cat("comparable across measures and unreadable on its own.\n")
cat("Standardising is useful for comparing across measures, and it hides the\n")
cat("units -- so report the raw difference too.\n")

A raw difference of 4.07 points against a pooled SD of 14.8 gives d = 0.275. Against a spread of 5 rather than 15, a raw difference of 7.91 gives d = 0.74: the raw gap grew by a factor of 1.9 and d by a factor of 2.7. The extra growth is entirely the change in spread.

The mistake this prevents

The mistake is quoting d as though it were a property of the treatment. It is a property of the treatment *and* the population's variability, so it moves when you change who you study.

Takeaway

Report the raw difference first and d alongside it when comparing across scales. Treat the small/medium/large labels as a last resort, and state the pooled SD so a reader can convert back.

Mark complete