Unit 03.02: Look at the shape before testing the means

Plot the distribution before you test the means. Two numbers cannot describe a distribution with two peaks.

Look at the shape first

Summary statistics compress a distribution to a centre and a spread, and neither can express bimodality, a floor effect, a spike at zero or a truncation. Groups with nearly identical means and standard deviations can have completely different shapes, and the shape is often the finding.

The charts do different jobs. A histogram shows one distribution's shape. Overlaid densities compare two. A box plot puts medians and spreads side by side. A scatter plot shows whether two variables move together.

It matters for test choice too: a t-test on a strongly bimodal variable tests a mean that describes nobody.

This block summarises two routes and then draws them.

import numpy as np, pandas as pd, tempfile, pathlib
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

rng = np.random.default_rng(31)
d = pd.DataFrame({
    "route": np.repeat(["A", "B"], 80),
    "minutes": np.concatenate([
        rng.normal(35, 4, 80),
        np.concatenate([rng.normal(29, 3, 40), rng.normal(41, 3, 40)]),
    ]),
})

print(d.groupby("route", observed=True)["minutes"]
      .agg(n="size", mean="mean", sd="std").round(2).to_string())
print()
print("The two means are close and the distributions are not the same shape:")
print("route B is bimodal, two clusters near 29 and 41, which no mean and")
print("standard deviation can express.")
print()
for name, question in {
    "histogram": "the shape of one distribution",
    "kde/density": "two shapes overlaid for comparison",
    "boxplot": "medians and spread side by side",
    "scatter": "whether two variables move together",
}.items():
    print(f"  {name:12s} {question}")

fig, ax = plt.subplots(figsize=(6, 3))
for route, grp in d.groupby("route", observed=True):
    ax.hist(grp.minutes, bins=20, alpha=0.6, label=route)
ax.set_xlabel("Minutes"); ax.set_ylabel("Count"); ax.legend()
out = pathlib.Path(tempfile.mkdtemp()) / "dists.png"
fig.savefig(out, dpi=100, bbox_inches="tight")
print(f"\nPlot the distributions before testing the means: {out.stat().st_size} bytes")

The means are 34.75 and 35.21 — close enough to look like the same distribution — with standard deviations of 3.64 and 6.88. Route B is actually bimodal, two clusters near 29 and 41, and no combination of mean and standard deviation can say so. A test comparing the means would answer a question about B that has no sensible answer.

The mistake this prevents

The mistake is going straight from groupby to ttest_ind. The summary table looks reasonable, the test runs, and nobody ever sees that one group is two populations.

Takeaway

Plot the distributions before testing. Use histograms or densities to check shape, and reconsider the method if a distribution is bimodal, truncated or heavily skewed.

Mark complete