Unit 09.06: One row can reverse the conclusion

One row out of forty-one can reverse your conclusion, and nothing in the summary output will mention it.

Leverage, influence, and Cook's distance

Leverage means an observation is unusual in its predictors — far out along x. Influence means removing it changes the fitted model. A point can have high leverage without being influential if it happens to sit on the line; the dangerous ones are high leverage *and* off the line.

Cook's distance, from model.get_influence().cooks_distance, combines both into one number per observation. A common rule of thumb flags values above 4/n, though the more useful signal is usually a value far larger than every other.

Finding an influential point is not permission to delete it. It is a prompt to find out what that observation is.

This block fits a line with one point placed far out in x and off the trend.

import numpy as np, pandas as pd
import statsmodels.formula.api as smf

rng = np.random.default_rng(607)
x = np.append(rng.uniform(1, 10, 40), 30.0)          # one point far out in x
y = np.append(5 + 2 * x[:40] + rng.normal(0, 3, 40), 12.0)   # and off the line
d = pd.DataFrame({"x": x, "y": y})

full = smf.ols("y ~ x", data=d).fit()
without = smf.ols("y ~ x", data=d.iloc[:40]).fit()

print(f"Slope with all {len(d)} points : {full.params['x']:.3f}")
print(f"Slope without the last one  : {without.params['x']:.3f}")
print(f"One row changed the slope by  {abs(full.params['x'] - without.params['x']):.2f}\n")

cooks = full.get_influence().cooks_distance[0]
order = np.argsort(cooks)[::-1]
print(f"Largest Cook's distance : {cooks[order[0]]:.3f} at observation {order[0]}")
print(f"Next largest            : {cooks[order[1]]:.3f}")
print(f"Rule of thumb 4/n       : {4 / len(d):.3f}")
print(f"Observations flagged    : {(cooks > 4 / len(d)).sum()}\n")

print("High leverage means unusual in x. Influential means removing it changes")
print("the model. Investigate such a point -- do not delete it because it is")
print("inconvenient. Report the model with and without it.")

The slope is 0.522 with all 41 points and 2.295 without the last one — a single row changed it by 1.77, more than reversing the conclusion. Its Cook's distance is 38.028 against a next-largest of 0.090, over four hundred times greater, and far above the 4/n threshold of 0.098. Exactly 1 observation is flagged.

The mistake this prevents

The mistake is deleting an inconvenient point because it is 'an outlier'. It may be the most informative observation you have, or a data entry error — and only investigation tells you which.

Takeaway

Compute Cook's distance for every regression and look at the largest values. Investigate influential points rather than removing them, and when one materially changes the result, report the model both with and without it.

Mark complete