Unit 08.01: Values or ranks

Pearson asks whether the relationship is a straight line. Spearman asks only whether it consistently goes one way.

Values or ranks

Pearson correlates the values, so it measures straight-line association and is sensitive to extreme observations. Spearman correlates the ranks, so it measures any consistently increasing or decreasing relationship and is much less affected by a single unusual point.

When a relationship is monotonic but curved, Spearman is higher — and the gap between the two is itself informative, since it says the relationship is real but not linear.

Either way, report the confidence interval. A correlation from a moderate sample is far less precisely determined than the three decimal places suggest.

This block correlates a curved monotonic relationship both ways, then adds a contradicting point.

set.seed(502)
x <- runif(60, 1, 10)
y <- exp(x / 3) + rnorm(60, 0, 1)      # monotonic but strongly curved

cat("Pearson  (straight-line):", round(cor(x, y, method = "pearson"), 3), "\n")
cat("Spearman (rank, monotonic):", round(cor(x, y, method = "spearman"), 3), "\n\n")

cat("Spearman is higher because the relationship always increases even\n")
cat("though it is not a straight line. Pearson only sees the straight part.\n\n")

# Robustness to a single extreme value.
xo <- c(x, 10.5); yo <- c(y, 2)
cat("After adding one contradicting point:\n")
cat("  Pearson :", round(cor(x, y), 3), "->", round(cor(xo, yo), 3), "\n")
cat("  Spearman:", round(cor(x, y, method = "spearman"), 3), "->",
    round(cor(xo, yo, method = "spearman"), 3), "\n\n")

ct <- cor.test(x, y)
cat("Pearson with a test: r =", round(ct$estimate, 3),
    " 95% CI [", round(ct$conf.int[1], 3), ",",
    round(ct$conf.int[2], 3), "]  p =", signif(ct$p.value, 3), "\n")
cat("Report the interval. With n = 60, r is not pinned down tightly.\n")

Pearson gives 0.935 and Spearman 0.955 on the curved relationship — Spearman higher, because the relationship always increases even though it is not a straight line. Adding one contradicting observation moves Pearson to 0.869 and Spearman to 0.88. The interval on Pearson runs from 0.894 to 0.961: even at r = 0.935 with n = 60, the coefficient is not pinned down to two decimals.

The mistake this prevents

The mistake is quoting a correlation to three decimals from a sample of thirty. The interval is usually wide enough to make the third decimal meaningless.

Takeaway

Use Pearson for linear relationships and Spearman when the relationship is monotonic but curved or the data has extreme values. Report the interval and treat a large Pearson–Spearman gap as evidence of curvature.

Mark complete