Unit 01.01: Reliability as a property of the whole path

Every stage can look healthy while the end-to-end experience is not, and the arithmetic is unforgiving.

Multiply, do not average

Success rates along a path multiply. Five stages at 95-99% each do not produce a 97% system.

The code multiplies through a five-stage path.

STAGES = [
    ("request received",  0.999),
    ("retrieval",         0.95),
    ("model call",        0.99),
    ("parse and validate", 0.97),
    ("action or display", 0.999),
]

end_to_end = 1.0
print(f"{'stage':20} {'succeeds':>9} {'cumulative':>11}")
for name, p in STAGES:
    end_to_end *= p
    print(f"{name:20} {p:>9.3f} {end_to_end:>11.3f}")

print(f"\nevery stage looks healthy; end to end is {end_to_end:.1%}")
print(f"that is {(1 - end_to_end) * 1000:.0f} failed requests per 1,000")

# No stage is below 95% and one request in fourteen fails. Reliability is a
# property of the path, and a dashboard showing five green stages can sit above
# a user experience nobody would call reliable.

No stage is below 95% and roughly one request in fourteen fails. A dashboard showing five green stages sits above a user experience nobody would call reliable.

The retrieval stage at 0.95 is doing most of the damage, which is the useful part: the arithmetic tells you where to spend, and it is rarely the stage that feels most fragile.

The mistake this prevents

The mistake is setting per-stage targets without computing the product. "Every stage above 95%" sounds rigorous and permits a 77% system. Set the end-to-end target first and derive the per-stage ones from it.

Takeaway

Reliability multiplies along the path. Set the end-to-end target and derive stage targets from it - per-stage targets set independently permit a system far worse than any of them.

Mark complete