Unit 09.04: A suite fast enough to run on every save

The suite that runs on every save is the one that catches most regressions.

Three hundred tests, two seconds

Five groups with counts and timings.

The code totals them.

SUITE = [
    ("model validation",        120, 0.002, "no network"),
    ("endpoint contract tests",  80, 0.008, "no network"),
    ("service logic",            95, 0.001, "no network"),
    ("database integration",     14, 0.180, "needs a database"),
    ("live model calls",          4, 2.400, "paid, slow"),
]
print(f"{'group':26} {'tests':>6} {'sec each':>9} {'total':>8}  note")
total = 0.0
for name, count, each, note in SUITE:
    seconds = count * each
    total += seconds
    print(f"{name:26} {count:>6} {each:>9.3f} {seconds:>7.2f}s  {note}")

offline = sum(c * e for _, c, e, n in SUITE if n == "no network")
offline_n = sum(c for _, c, _, n in SUITE if n == "no network")
all_n = sum(c for _, c, _, _ in SUITE)
print(f"\noffline: {offline_n}/{all_n} tests in {offline:.2f}s")
print(f"the {all_n - offline_n} others are {(total - offline) / total:.0%} of the time")

# Keep the offline group under a couple of seconds and it runs on every save.
# Put the rest behind a marker that runs before a release.

Nearly all the tests need no network and take a fraction of the wall clock. The handful that need a database or a live model take most of it.

Splitting them means the fast group runs on every save and the slow group runs before a release - and the fast group is where model validation, endpoint contracts and service logic all live.

The mistake this prevents

The mistake is one command that runs everything. The suite then takes minutes, developers run it less, and the cheap tests that would have caught the regression stop running at the moment they were needed.

Takeaway

Split offline tests from those needing a network and keep the offline group under a couple of seconds. It is where most of the coverage is.

Mark complete