Skip to course content
Free LLMOps course

LLMOps for Reliable AI Applications

Unit 10.04: Gates that people cannot quietly skip

A gate nobody can skip gets removed. A gate anyone can skip silently gets ignored. The one that survives is in between.

Overridable, but on the record

Five gate designs, sorted by whether they survive contact with a deadline.

The code compares all five.

DESIGNS = [
    ("gate runs in CI, blocks the merge",           "hard", "cannot skip silently"),
    ("gate runs in CI, failure is a warning",       "soft", "ignored within a month"),
    ("someone runs the eval before release",        "none", "skipped under deadline"),
    ("gate can be overridden with a recorded reason", "hard", "skippable, but visibly"),
    ("gate runs nightly, results emailed",          "soft", "nobody reads it"),
]
print(f"{'design':46} {'strength':9} outcome")
for design, strength, outcome in DESIGNS:
    print(f"{design:46} {strength:9} {outcome}")

print("""
The fourth is the one to build. An unskippable gate gets removed the first time
it blocks something genuinely urgent; a gate with a recorded override survives,
because the override is available and leaves a trail.

Review the overrides monthly. A gate overridden every week is a bar set wrong,
and that is worth knowing rather than tolerating.
""")

The design to build is the fourth: an override that requires a recorded reason. An unskippable gate gets removed the first time it blocks something genuinely urgent, and the removal is permanent.

The recorded override is also a measurement. Review the overrides monthly: a gate overridden every week is a bar set wrong, and that is worth discovering from the log rather than from the gate quietly ceasing to matter.

The mistake this prevents

The mistake is the fifth design - a nightly run emailing results. It feels like a gate and gates nothing. Nobody reads it after the second week, and a regression sits in the inbox for a month.

Takeaway

Make gates overridable with a recorded reason, and review the overrides monthly. Unskippable gates get deleted; silent ones get ignored; recorded overrides survive and tell you when a bar is wrong.