Unit 01.01: Every call costs money and can fail

Every call costs money and can fail, and one of the failure modes charges you without telling you whether it worked.

Five outcomes, three of them not an answer

Success, rate limit, server error, timeout, and content filtering - with what each costs.

The code prices them.

CALLS = [
    ("ok",              200, 0.0021, "usable"),
    ("rate limited",    429, 0.0,    "retry with backoff"),
    ("server error",    500, 0.0,    "retry, then fall back"),
    ("timeout",         None, 0.0021, "may have succeeded upstream"),
    ("content filtered", 200, 0.0021, "charged, no usable output"),
]
print(f"{'outcome':18} {'status':>7} {'cost':>8}  what you do")
for outcome, status, cost, action in CALLS:
    print(f"{outcome:18} {str(status):>7} {cost:>8.4f}  {action}")

billed = sum(c for _, _, c, _ in CALLS)
useful = sum(c for o, _, c, _ in CALLS if o == "ok")
print(f"\nbilled ${billed:.4f}, useful ${useful:.4f} "
      f"({useful / billed:.0%} of spend produced an answer)")

# The timeout row is the one that matters operationally: you were charged, the
# call may have succeeded, and you cannot tell. Anything with a side effect
# behind such a call needs an idempotency key.

The timeout row is the operationally important one: you were charged, the call may have succeeded upstream, and you cannot tell from your side. Anything with a side effect behind such a call needs an idempotency key.

The filtered row is the other surprise - a successful HTTP call, a full charge, and no usable output.

The mistake this prevents

The mistake is measuring cost per call rather than per successful answer. A 20% retry rate is a 20% cost increase that shows up on the invoice as a mystery rather than as the quality problem it is.

Takeaway

Calls cost money whether or not they produce an answer. Measure cost per successful result, and put an idempotency key behind anything a timeout could have half-completed.

Mark complete