Unit 09.00: Every slope is now conditional

Adding a second predictor changes what the first one means. That is the point, and it is where most misreadings start.

Every slope is now conditional

smf.ols('y ~ a + b') estimates a slope for each predictor, and each is the association with y holding the other fixed. In a simple regression the slope carries everything correlated with that predictor; in a multiple regression it carries only what is left once the others are accounted for.

The same variable can therefore have different coefficients in two models and both be correct — they answer different questions.

Adding a genuinely relevant predictor also shrinks the residual spread, which is the practical benefit: the model's typical error falls.

This block fits price on size, then on size and age.

import numpy as np, pandas as pd
import statsmodels.formula.api as smf

rng = np.random.default_rng(601)
d = pd.DataFrame({"size": rng.uniform(40, 200, 200),
                  "age": rng.uniform(0, 60, 200)})
d["price"] = 50 + 2.1 * d["size"] - 0.8 * d["age"] + rng.normal(0, 25, 200)

simple = smf.ols("price ~ size", data=d).fit()
multiple = smf.ols("price ~ size + age", data=d).fit()

print("price ~ size")
print(simple.params.round(3).to_string())
print("\nprice ~ size + age")
print(multiple.params.round(3).to_string())
print()
print("`y ~ a + b` reads: model y as a function of a AND b. Each slope is now")
print("the association with y HOLDING THE OTHER FIXED.\n")
print(f"R-squared  : {simple.rsquared:.4f} -> {multiple.rsquared:.4f}")
print(f"Residual SD: {np.sqrt(simple.scale):.2f} -> {np.sqrt(multiple.scale):.2f}")
print(f"Adding a genuinely relevant predictor cut the typical error by"
      f" {1 - np.sqrt(multiple.scale) / np.sqrt(simple.scale):.0%}.")

The size slope is 2.085 alone and 2.065 with age included — barely changed, because size and age were generated independently. Age enters at −0.741. R-squared rises from 0.9206 to 0.9388 and the residual SD falls from 28.62 to 25.18, cutting the typical error by 12%. When predictors are uncorrelated, adding one leaves the others alone; when they are correlated, it will not.

The mistake this prevents

The mistake is quoting a multiple-regression coefficient without the 'holding the others fixed' clause. It is a conditional quantity, and dropping the condition changes what the sentence claims.

Takeaway

State which variables are in the model whenever you quote a coefficient, and say 'adjusted for' explicitly. Report the residual SD alongside R-squared, because it is in the outcome's own units.

Mark complete