Unit 06.03: Showing the spread rather than asserting it

Two bars with means on them assert a separation the data may not show.

Show the overlap

A control and a treated group differing by half a standard deviation.

The code reports the difference and the overlap.

import numpy as np

rng = np.random.default_rng(3)
control = rng.normal(100, 12, 40)
treated = rng.normal(106, 12, 40)

print(f"control mean {control.mean():.1f}, treated mean {treated.mean():.1f}")
print(f"difference in means: {treated.mean() - control.mean():+.1f}")
print(f"standard deviation of each group: ~{control.std():.0f}")
print(f"\noverlap: {(control > treated.mean()).mean():.0%} of control values "
      f"are above the treated MEAN")

print("""
The difference in means is real and small relative to the spread. Two bars
with the means on them assert a clean separation that the data does not show.

Draw the points. The reader then sees both the difference and how much the
groups overlap, and forms their own view of how much to trust it.
""")

The difference in means is real. A large share of the control group still sits above the treated group's mean, which is what "real but small relative to spread" looks like.

Two bars communicate none of that. Drawing the points lets the reader see both the difference and the overlap, and form their own view about how much to rely on it.

The mistake this prevents

The mistake is adding an error bar and considering the problem solved. An error bar shows the uncertainty of the *mean*, not the spread of the *data* - they differ by a factor of the square root of n, and readers routinely confuse them.

Takeaway

Draw the points when comparing two groups. Bars with means assert a clean separation, and error bars describe the mean rather than the spread.

Mark complete