Unit 08.05: Two different jobs for the same equation

A model that predicts well can be completely wrong about why, and which kind of wrong you can tolerate depends on the question.

Two different jobs for the same equation

For prediction the only thing that matters is whether the forecasts are accurate. A predictor that is merely correlated with the outcome through some third variable is perfectly acceptable.

For explanation you want to know what would happen if you intervened, and now a merely correlated predictor is actively misleading. A confounder — something that causes both variables — produces a strong, significant, entirely spurious relationship.

Adding the confounder to the model usually makes the spurious relationship collapse, which is a useful diagnostic. It only works for confounders you thought of and measured, which is why observational explanation is hard and randomisation is valuable.

This block regresses drownings on ice cream sales, then adds temperature.

set.seed(506)
n <- 200
temperature <- runif(n, 5, 35)
ice_cream   <- 20 + 3 * temperature + rnorm(n, 0, 10)
drownings   <- 2 + 0.30 * temperature + rnorm(n, 0, 1.5)

cat("Correlation, ice cream and drownings:",
    round(cor(ice_cream, drownings), 3), "\n")
naive <- lm(drownings ~ ice_cream)
cat("Slope on ice cream alone :", round(coef(naive)[2], 4),
    "  p =", signif(summary(naive)$coefficients[2, 4], 3), "\n\n")

adjusted <- lm(drownings ~ ice_cream + temperature)
cat("With temperature in the model:\n")
cat("  ice cream slope:", round(coef(adjusted)[2], 4),
    "  p =", signif(summary(adjusted)$coefficients[2, 4], 3), "\n")
cat("  temperature slope:", round(coef(adjusted)[3], 4),
    "  p =", signif(summary(adjusted)$coefficients[3, 4], 3), "\n\n")

cat("The ice cream effect vanishes once temperature is accounted for.\n")
cat("For PREDICTION the naive model is fine -- ice cream sales genuinely\n")
cat("forecast drownings, R-squared", round(summary(naive)$r.squared, 3), ".\n")
cat("For EXPLANATION it is worthless, and banning ice cream would save\n")
cat("nobody. Which question you are answering decides which model is wrong.\n")

Ice cream sales and drownings correlate at 0.812, and the naive regression gives a slope of 0.0839 with p = 2.76e-48 — about as significant as results get. Adding temperature, the ice cream slope falls to −0.0102 with p = 0.329, while temperature takes a slope of 0.3278. The relationship was entirely temperature. For prediction the naive model is genuinely useful — R-squared 0.66, ice cream sales do forecast drownings — and for explanation it is worthless.

The mistake this prevents

The mistake is reading a significant coefficient from observational data as a cause. The p-value measures evidence against zero association, and a confounded association is a real association.

Takeaway

Decide whether you are predicting or explaining before choosing a model. For explanation, list the plausible confounders and include the ones you have, and say plainly which ones you could not measure.

Mark complete