Unit 01.04: The failure you have already accepted

Every system has failures it will not fix. Writing them down is what separates a decision from an oversight.

Failure, mitigation, residual risk

Three columns. The third is what makes the row honest - a mitigation without a residual claims the problem is solved.

The code lists four accepted failures.

ACCEPTED = [
    ("paraphrases far from document wording may miss",
     "logged and reviewed weekly", "medium"),
    ("staleness surfaced but not auto-detected",
     "updated date shown in every answer", "medium"),
    ("English only", "non-English queries refuse rather than guess", "low"),
    ("citation check is term overlap, not entailment",
     "sampled manual review of 20 answers a week", "medium"),
]

print(f"{'accepted failure':52} {'mitigation':38} residual")
for failure, mitigation, residual in ACCEPTED:
    print(f"{failure:52} {mitigation:38} {residual}")

print(f"""
{len(ACCEPTED)} failures you have decided to live with. Writing them down converts
an unknown unknown into a documented trade-off someone can disagree with.

A system presented as having no known failure modes reads to an experienced
reviewer as an analysis that was never done.
""")

"Citation check is term overlap, not entailment" with a mitigation of twenty sampled reviews a week and a residual of medium is a complete row. A reviewer can push back on the sample size, the frequency, or the judgement that medium is acceptable.

That is the point. Each row converts something a reviewer would otherwise have to discover into something they can disagree with before it reaches production.

The mistake this prevents

The mistake is presenting a system as having no known failure modes. It reads as thoroughness and is read by experienced reviewers as inexperience - they know the failures exist, so a list of none means the analysis was not done and they now have to do it themselves.

Takeaway

Name the failures you accept, each with a mitigation and a residual risk. A documented trade-off is reviewable; an undocumented one is discovered by a user.

Mark complete