Unit 11.03: Tracking whether the fix held

"Deployed" and "fixed" are different states, and almost every write-up conflates them.

Four weeks at baseline

The metric tracked from the incident through recovery and beyond.

The code shows five weeks.

WEEKS = [
    ("week 0 (incident)", 0.84, "rollback applied"),
    ("week 1", 0.96, "fix deployed with q42 in the gate"),
    ("week 2", 0.96, ""),
    ("week 3", 0.95, ""),
    ("week 4", 0.96, "fix confirmed held"),
]
print(f"{'when':20} {'grounding':>10}  note")
for when, value, note in WEEKS:
    print(f"{when:20} {value:>10.2f}  {note}")

print("""
Four weeks of the metric back at baseline is what "fixed" means. A single green
run the day after is what "deployed" means, and the two get conflated in almost
every incident write-up.

Set a review date when you close the incident, and put the metric on it.
""")

One green run the day after the fix is what "deployed" means. Four weeks of the metric back at baseline is what "fixed" means, and the gap between them is where a fix that only worked under low traffic, or only for the case you tested, quietly fails.

Setting the review date when you close the incident is the mechanism. Without one, nobody checks - the incident is closed, attention has moved, and the regression is discovered by the next incident.

The mistake this prevents

The mistake is closing the incident when the fix deploys. That is the point of maximum confidence and minimum evidence: the fix has run once, under observation, on the case you constructed to test it.

Takeaway

Set a review date when closing an incident and check the metric on it. Deployed means it ran once; fixed means the metric held for weeks.

Mark complete