Unit 09.05: Look for pattern, not for size

R-squared cannot tell you the model is the wrong shape. The residuals can, and a badly wrong model often has the higher R-squared.

Look for pattern, not for size

Residuals from a correctly specified model scatter around zero with no systematic relationship to the fitted values. A pattern in them is the model telling you something is missing.

A straight line fitted to a curved relationship produces the classic signature: too high at the extremes and too low in the middle, or the reverse. The mean residual within thirds of the fitted range makes this visible as a sign change.

The three plots worth drawing are residuals against fitted for shape, a Q-Q plot for the tails, and the square root of the absolute residuals against fitted for constant spread.

This block fits a line to correctly linear data and to genuinely curved data.

import numpy as np, pandas as pd
import statsmodels.formula.api as smf

rng = np.random.default_rng(606)
d = pd.DataFrame({"x": rng.uniform(1, 20, 150)})
d["y_ok"] = 5 + 2 * d.x + rng.normal(0, 4, 150)
d["y_curve"] = 5 + 2 * d.x + 0.35 * d.x ** 2 + rng.normal(0, 4, 150)

# Split the fitted values into thirds and look at the MEAN residual in each.
# A correct model scatters around zero everywhere. A line fitted to a curve
# is too low at the ends and too high in the middle, or the reverse.
for col in ("y_ok", "y_curve"):
    m = smf.ols(f"{col} ~ x", data=d).fit()
    thirds = pd.qcut(m.fittedvalues, 3, labels=["low", "mid", "high"])
    means = m.resid.groupby(thirds, observed=True).mean()
    print(f"{col:8s} R2 {m.rsquared:.3f}   mean residual by fitted third:"
          f" {means['low']:+6.2f} {means['mid']:+6.2f} {means['high']:+6.2f}")

print("\nBoth report a high R-squared -- the curved one is HIGHER.")
print("Only the residuals show that it is the wrong shape: its mean residual")
print("swings from positive to negative and back, while the correct model's")
print("stays near zero across the whole fitted range.\n")
print("Plot resid against fittedvalues for shape, a Q-Q plot for the tails,")
print("and sqrt(|resid|) against fitted for constant spread.")

The correct model has R-squared 0.886 and mean residuals of +0.51, −1.12, +0.61 across the fitted thirds — noise. The curved model has the *higher* R-squared at 0.958, and its mean residuals run +4.85, −10.24, +5.39: a systematic swing of fifteen units. A reader comparing R-squared alone would pick the wrong model.

The mistake this prevents

The mistake is selecting a model on R-squared and never plotting the residuals. R-squared measures how much variance is explained, not whether the functional form is right.

Takeaway

Plot residuals against fitted values for every model you take seriously. Treat any systematic pattern as evidence of a missing term or a wrong shape, and never compare models on R-squared alone.

Mark complete