Unit 08.04: The mean response and a new observation

There are two intervals in a regression, they differ by a factor of several, and they answer different questions.

The mean response and a new observation

Every coefficient has a confidence interval, and it should be reported instead of — or at least alongside — its p-value, because it says how precisely the slope is known.

For predictions there are two intervals. A confidence interval for the mean response answers 'where is the average outcome at this x?' A prediction interval answers 'where will a single new observation fall?', and it is much wider, because a new point carries the residual scatter as well as the uncertainty in the line.

Quoting the confidence interval when someone asked about an individual case understates the uncertainty dramatically.

This block reports the coefficient intervals, then both prediction intervals at one x.

suppressPackageStartupMessages(library(broom))
set.seed(505)
x <- runif(45, 0, 10)
y <- 3 + 1.5 * x + rnorm(45, 0, 5)
model <- lm(y ~ x)

print(tidy(model, conf.int = TRUE)[, c("term", "estimate", "std.error",
                                       "conf.low", "conf.high", "p.value")])

ci <- confint(model)["x", ]
cat("\nSlope 95% CI: [", round(ci[1], 3), ",", round(ci[2], 3), "]\n")
cat("Width       :", round(diff(ci), 3), "\n")
cat("Excludes zero:", ci[1] * ci[2] > 0, "\n\n")

# Prediction intervals are much wider than confidence intervals.
new <- data.frame(x = 5)
conf <- predict(model, new, interval = "confidence")
pred <- predict(model, new, interval = "prediction")
cat("At x = 5, fitted value", round(conf[1], 2), "\n")
cat("  CI for the MEAN response  : [", round(conf[2], 2), ",",
    round(conf[3], 2), "]  width", round(conf[3] - conf[2], 2), "\n")
cat("  PI for a NEW observation  : [", round(pred[2], 2), ",",
    round(pred[3], 2), "]  width", round(pred[3] - pred[2], 2), "\n")
cat("\nThe prediction interval is wider because a new point carries the\n")
cat("residual scatter as well as the uncertainty in the line.\n")

The slope is 1.46 with an interval from 0.953 to 1.96 — width 1.007, excluding zero, so the relationship is established while its magnitude is known only to within a factor of two. At x = 5 the fitted value is 10.66. The interval for the *mean* response runs from 9.16 to 12.16, width 3; the interval for a *new observation* runs from 0.66 to 20.66, width 20.01 — nearly seven times wider.

The mistake this prevents

The mistake is answering 'what will this customer spend?' with a confidence interval. That interval is for the average customer at that x, and it is far narrower than the range an individual will fall in.

Takeaway

Report coefficient intervals rather than p-values alone. Choose deliberately between a confidence interval and a prediction interval, and say in the text which one a figure shows.

Mark complete