Unit 05.04: Four questions choose the scipy call

Four questions choose the test, and each maps to a specific scipy call.

Outcome type, group count, design, assumptions

What type is the outcome — numeric, binary, categorical? How many groups — one against a target, two, or more? Is the design paired or independent? And do the method's assumptions hold?

Answering those four picks the function from a short table. Memorising function names without the questions is what produces t-tests on yes/no outcomes.

The paired/independent distinction is not a technicality. Pairing removes between-unit variation from the comparison, and when that variation is large relative to the effect, ttest_rel finds what ttest_ind has no chance of seeing.

This block lays out the table and then demonstrates why pairing is its own row.

import numpy as np
from scipy import stats

table = [
    ("numeric", "1",  "-",           "stats.ttest_1samp(x, popmean=)"),
    ("numeric", "2",  "independent", "stats.ttest_ind(a, b, equal_var=False)"),
    ("numeric", "3+", "independent", "stats.f_oneway(*groups)"),
    ("numeric", "2",  "paired",      "stats.ttest_rel(after, before)"),
    ("binary",  "2",  "independent", "proportions_ztest / chi2_contingency"),
    ("binary",  "2+", "independent", "stats.chi2_contingency(table)"),
]
for outcome, groups, design, call in table:
    print(f"{outcome:8s} {groups:3s} {design:12s} -> {call}")

print("\nWhy 'paired' is a separate row, not a detail:")
rng = np.random.default_rng(205)
before = rng.normal(50, 10, 25)
after = before + rng.normal(3, 2, 25)      # each person improves by about 3

print(f"  independent-samples p = {stats.ttest_ind(after, before, equal_var=False).pvalue:.3g}")
print(f"  paired-samples      p = {stats.ttest_rel(after, before).pvalue:.3g}")
print("\nSame numbers. The paired test removes the between-person variation")
print("and finds an effect the independent test cannot see at all.")

Six rows cover most beginner work, each naming the scipy call. Then the same 25 before-and-after measurements are tested both ways: independent samples give p = 0.478 — nothing — while paired gives p = 3.35e-05. Identical numbers, and the choice of design decides whether a real, consistent three-point improvement is visible at all.

The mistake this prevents

The mistake is running ttest_ind on before-and-after data. It throws away the pairing, so the between-person spread swamps the within-person change and the effect disappears.

Takeaway

Answer the four questions in writing before choosing a test. Check explicitly whether the two measurements come from the same units, because that single fact can decide the result.

Mark complete