Unit 08.03: The residuals are where the model tells you about itself

The residuals are where the model tells you about itself. R-squared is the least informative number in the output.

Fitted, residual, and what R-squared is not

For each observation the model produces a fitted value and a residual — the gap between what happened and what the line predicted. Residuals always sum to zero by construction, so their *pattern*, not their total, is what carries information.

The residual standard deviation says how far a typical point sits from the line, in the outcome's own units. It is more useful than R-squared because it is interpretable.

R-squared is the share of variance in y that the line accounts for. A high value does not mean the model is correct — a badly curved relationship can produce one — and a low value does not mean there is no relationship, only that y varies a great deal around it.

This block reports the coefficients, three individual fits and the variance decomposition.

set.seed(504)
x <- runif(60, 0, 20)
y <- 5 + 2.2 * x + rnorm(60, 0, 6)
model <- lm(y ~ x)

cat("Intercept:", round(coef(model)[1], 3), "\n")
cat("Slope    :", round(coef(model)[2], 3), "\n\n")

fitted_vals <- fitted(model)
resid_vals  <- residuals(model)

cat("First three observations:\n")
for (i in 1:3) {
  cat(sprintf("  x = %5.2f  actual %6.2f  fitted %6.2f  residual %+6.2f\n",
              x[i], y[i], fitted_vals[i], resid_vals[i]))
}

cat("\nResiduals sum to (essentially) zero:",
    format(sum(resid_vals), scientific = TRUE), "\n")
cat("Residual SD (sigma):", round(summary(model)$sigma, 2), "\n\n")

cat("R-squared:", round(summary(model)$r.squared, 4), "\n")
cat("  = share of the variance in y the line accounts for\n")
cat("  = 1 -", round(var(resid_vals) / var(y), 4), "\n\n")
cat("A high R-squared does not mean the model is right, and a low one does\n")
cat("not mean the relationship is absent -- only that y varies a lot around it.\n")

The line is 1.652 + 2.431x. The first three observations show residuals of +2.28, +0.17 and +6.98 — the third point sits nearly seven units above the line. The residuals sum to 2.5e-14, which is zero to rounding, and their standard deviation is 5.88 in the units of y. R-squared is 0.8501, confirmed as 1 − 0.1499, the share of variance the residuals do not account for.

The mistake this prevents

The mistake is judging a model by R-squared alone. It rises whenever the outcome has more variance to explain, and it says nothing about whether the line is the right shape.

Takeaway

Read the residual standard deviation as the practical accuracy of the model, and inspect residual patterns rather than their total. Treat R-squared as one summary among several, never as a verdict.

Mark complete