Unit 12.02: One comparison, three views

The table, the test and the chart must describe the same comparison at the same grain — and the chart usually shows what the difference hides.

One comparison, three views

A common failure is that the table summarises one thing, the test compares another, and the chart shows a third. A reader who notices loses confidence in all three; a reader who does not is misled.

The chart's job is not to repeat the table. It is to show what the summary cannot: the overlap between groups, the shape of each distribution, the points that do not fit.

A significant difference between two heavily overlapping distributions is a real finding and a modest one, and only the chart conveys that second part.

This block produces the table, the test and the chart for one comparison.

import numpy as np, pandas as pd, tempfile, pathlib
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from scipy import stats

rng = np.random.default_rng(902)
d = pd.DataFrame({
    "arm": np.repeat(["A", "B"], 300),
    "value": np.concatenate([rng.normal(50, 9, 300), rng.normal(53, 9, 300)]),
})

tab = (d.groupby("arm", observed=True)["value"]
       .agg(n="size", mean="mean", sd="std")
       .assign(se=lambda x: x["sd"] / np.sqrt(x["n"])).round(2))
print(tab.to_string())

a = d.value[d.arm == "A"]
b = d.value[d.arm == "B"]
res = stats.ttest_ind(b, a, equal_var=False)
ci = res.confidence_interval()
print(f"\nDifference (B - A): {b.mean() - a.mean():.2f}"
      f"   95% CI [{ci.low:.2f}, {ci.high:.2f}]\n")

fig, ax = plt.subplots(figsize=(5, 3.5))
ax.boxplot([a, b], tick_labels=["A", "B"], widths=0.5)
ax.set_ylabel("Value")
ax.set_title("B scores higher, with heavily overlapping distributions")
out = pathlib.Path(tempfile.mkdtemp()) / "capstone.png"
fig.savefig(out, dpi=150, bbox_inches="tight")

print("Table, test and chart describe the same comparison at the same grain.")
print(f"The boxplot shows the overlap the difference hides: SDs of"
      f" {tab.sd['A']} and {tab.sd['B']}")
print(f"against a difference of {b.mean() - a.mean():.2f}.")
print(f"Chart written: {out.stat().st_size} bytes")

The table gives n = 300 per arm, means of 49.14 and 52.38, and standard deviations of 8.79 and 9.25. The test reports a difference of 3.24 with an interval from 1.79 to 4.69 — clearly non-zero. The boxplot then shows what the difference alone hides: the group SDs are around 9 against a difference of 3.24, so the two distributions overlap heavily. Real, reliable, and small relative to individual variation.

The mistake this prevents

The mistake is a chart showing only the two means with error bars. It repeats the table and conceals the overlap, which is the thing a reader most needs to see.

Takeaway

Make the table, the test and the chart describe the same comparison at the same grain. Choose a chart that shows the distributions, not just the summary, and state the group SDs beside the difference.

Mark complete