Unit 08.04: The mean response and a new observation

There are two intervals in a regression, they differ by a factor of several, and they answer different questions.

The mean response and a new observation

Every coefficient has a confidence interval, available from model.conf_int(), and it should be reported instead of — or at least alongside — its p-value, because it says how precisely the slope is known.

For predictions there are two intervals, and get_prediction(...).summary_frame() gives both. mean_ci_lower/upper answers 'where is the average outcome at this x?'. obs_ci_lower/upper answers 'where will a single new observation fall?', and is much wider, because a new point carries the residual scatter as well as the uncertainty in the line.

Quoting the confidence interval when someone asked about an individual case understates the uncertainty dramatically.

This block reports the coefficient intervals, then both prediction intervals at one x.

import numpy as np, pandas as pd
import statsmodels.formula.api as smf

rng = np.random.default_rng(505)
d = pd.DataFrame({"x": rng.uniform(0, 10, 45)})
d["y"] = 3 + 1.5 * d["x"] + rng.normal(0, 5, 45)
model = smf.ols("y ~ x", data=d).fit()

print(model.summary2().tables[1].round(4).to_string())
lo, hi = model.conf_int().loc["x"]
print(f"\nSlope 95% CI  : [{lo:.3f}, {hi:.3f}]   width {hi - lo:.3f}")
print(f"Excludes zero : {lo * hi > 0}\n")

pred = model.get_prediction(pd.DataFrame({"x": [5.0]})).summary_frame(alpha=0.05)
row = pred.iloc[0]
print(f"At x = 5, fitted value {row['mean']:.2f}")
print(f"  CI for the MEAN response : [{row['mean_ci_lower']:.2f},"
      f" {row['mean_ci_upper']:.2f}]   width"
      f" {row['mean_ci_upper'] - row['mean_ci_lower']:.2f}")
print(f"  PI for a NEW observation : [{row['obs_ci_lower']:.2f},"
      f" {row['obs_ci_upper']:.2f}]   width"
      f" {row['obs_ci_upper'] - row['obs_ci_lower']:.2f}")
print("\nThe prediction interval is wider because a new point carries the")
print("residual scatter as well as the uncertainty in the line.")

The slope is 1.5911 with an interval from 0.939 to 2.244 — width 1.305, excluding zero, so the relationship is established while its magnitude is known only to within a factor of about two. At x = 5 the fitted value is 12.00. The interval for the *mean* response runs from 10.09 to 13.92, width 3.83; the interval for a *new observation* runs from −0.48 to 24.49, width 24.96 — more than six times wider.

The mistake this prevents

The mistake is answering 'what will this customer spend?' with a confidence interval. That interval is for the average customer at that x, and it is far narrower than the range an individual will fall in.

Takeaway

Report coefficient intervals rather than p-values alone. Choose deliberately between mean_ci and obs_ci, and say in the text which one a figure shows.

Mark complete