Unit 10.00: Defining what must pass to ship

A gate names a metric, a bar, a comparison and a severity. Anything less is a suggestion.

Eight gates, seven of them blockers

Each row is machine-checkable, which is what lets it run in CI rather than in a meeting.

The code lists the full gate set.

GATES = [
    ("approval violations",   "count", 0,     "==", "blocker"),
    ("format failure rate",   "rate",  0.01,  "<=", "blocker"),
    ("correctness (40 cases)", "rate", 0.90,  ">=", "blocker"),
    ("grounding rate",        "rate",  0.95,  ">=", "blocker"),
    ("refusal accuracy",      "rate",  0.95,  ">=", "blocker"),
    ("p95 latency",           "ms",    2000,  "<=", "blocker"),
    ("cost per request",      "usd",   0.005, "<=", "warning"),
    ("newly broken cases",    "count", 0,     "==", "blocker"),
]
print(f"{'gate':26} {'kind':6} {'bar':>8} {'op':>3}  severity")
for name, kind, bar, op, severity in GATES:
    print(f"{name:26} {kind:6} {bar:>8} {op:>3}  {severity}")

blockers = sum(1 for *_, s in GATES if s == "blocker")
print(f"\n{blockers} blockers, {len(GATES) - blockers} warning")
print("every gate names a metric, a bar, a comparison and a severity")

# "Newly broken cases == 0" is the gate that catches the change that improves
# the average and breaks a case that used to work.

"Newly broken cases == 0" is the gate that catches what averages hide: a change that improves the overall score while breaking a case that used to work. Without it, every gate can pass on a release that made things worse for some users.

Cost is a warning rather than a blocker, which is a deliberate choice. A release that improves quality and costs slightly more should be a conversation, not an automatic block - and marking it as a warning is how you say that in the config rather than in someone's head.

The mistake this prevents

The mistake is setting every gate as a blocker because everything matters. A gate set with no warnings gets overridden routinely, and once overriding is routine the blockers stop meaning anything either.

Takeaway

Every gate needs a metric, bar, comparison and severity. Include newly-broken-cases at zero, and use warnings deliberately so blockers keep their force.

Mark complete