Unit 06.03: The test detects; the residuals explain

chi2_contingency tells you two categorical variables are associated. The residuals tell you how — and on a 2×2 table it quietly applies a correction you may not want.

The test detects; the residuals explain

A chi-square test compares observed counts with the counts expected if the two variables were unrelated. A large statistic means the table does not look like independence.

That is all it says: no direction, no effect size, no indication of which cells are responsible. The standardised residuals — observed minus expected, over the square root of expected — supply all three.

Two practical notes. Expected counts should not be too small; the usual rule of thumb wants every one above 5, and below that Fisher's exact test is the right tool. And for a 2×2 table scipy applies Yates' continuity correction by default, which is more conservative than most people expect — correction=False turns it off.

This block tests plan against churn and inspects the residuals.

import numpy as np
from scipy import stats

observed = np.array([[120, 80], [95, 105], [60, 140]])
rows = ["basic", "standard", "premium"]
cols = ["retained", "churned"]

print("Observed:")
print(f"{'':10s}" + "".join(f"{c:>10s}" for c in cols))
for r, row in zip(rows, observed):
    print(f"{r:10s}" + "".join(f"{v:10d}" for v in row))

chi2, p, dof, expected = stats.chi2_contingency(observed)
print(f"\nchi2 = {chi2:.3f}   dof = {dof}   p = {p:.3e}\n")

print("Expected under independence:")
for r, row in zip(rows, expected):
    print(f"{r:10s}" + "".join(f"{v:10.1f}" for v in row))

residuals = (observed - expected) / np.sqrt(expected)
print("\nStandardised residuals -- where the association actually is:")
for r, row in zip(rows, residuals):
    print(f"{r:10s}" + "".join(f"{v:+10.2f}" for v in row))

print(f"\nSmallest expected count: {expected.min():.1f}"
      " (the rule of thumb wants every one above 5)")
print("\nThe test says plan and churn are associated. It does not say how.")
print("The residuals do: premium churns far more than independence predicts.")
print("Note: for a 2x2 table scipy applies Yates' correction by default --")
print("pass correction=False to turn it off.")

The test gives chi2 = 36.587 on 2 degrees of freedom, p = 1.135e-08 — plan and churn are associated. The residuals say how: premium sits at +3.04 on churned and −3.31 on retained, basic at −2.72 and +2.96. Premium churns far more than independence predicts and basic far less. The smallest expected count is 91.7, comfortably above the rule of thumb.

The mistake this prevents

The mistake is reporting the p-value and stopping. It says only that something is going on, and every actionable part of the finding is in the residuals.

Takeaway

Report the test, the expected counts and the standardised residuals together. Check the smallest expected count, switch to fisher_exact when it falls below about 5, and decide deliberately about Yates' correction on 2×2 tables.

Mark complete