Unit 09.01: Loops, retries, and runaway spend

A loop with no ceiling does not cost twice as much. It costs whatever the provider will serve before someone notices.

The ceiling is the control

A hop counter checked every iteration, with a hard stop. Without it, the loop's cost is bounded only by rate limits.

The code runs a loop with and without a ceiling.

MAX_HOPS = 6
TOKENS_PER_HOP = 3_000
RATE = 0.003 / 1000

hops = 0
spend = 0.0
log = []
while hops < 20:
    hops += 1
    spend += TOKENS_PER_HOP * RATE
    if hops > MAX_HOPS:
        log.append(f"hop {hops}: CEILING EXCEEDED -- run terminated")
        break
    log.append(f"hop {hops}: ${spend:.4f} spent")

for line in log:
    print(line)

print(f"\nwith a ceiling  : {hops} hops, ${spend:.3f}")
print(f"without a ceiling: 20 hops, ${20 * TOKENS_PER_HOP * RATE:.3f} "
      f"({20 / hops:.1f}x) -- and 20 is only where this loop stopped counting")

# A loop with no ceiling does not cost twice as much. It costs whatever the
# provider's rate limit permits before someone notices, which is usually the
# bill.

The ceiling stops it at seven hops. Without one, the twenty hops shown are not the loop's cost - they are where the example stopped counting, and a real loop continues until a rate limit, a timeout, or a person intervenes.

Overnight is the case to think about. A loop that starts at 6pm on a Friday and is noticed on Monday has had the whole weekend at whatever throughput the provider allows.

The mistake this prevents

Cost also grows without any loop at all. Each handoff typically carries the accumulated context forward, so a five-hop chain re-sends the conversation five times and each hop is more expensive than the last. That growth has a quality cost too: the material that actually matters occupies a smaller share of a larger context, and attention on it thins. Trim the handoff to an allowlist of fields, and measure tokens per hop rather than only per run.

The mistake is relying on a wall-clock timeout instead of a hop ceiling. A timeout generous enough for a legitimate long run is generous enough for several hundred loop iterations, and it tells you nothing about why the run ended.

Takeaway

Put a hop ceiling on every loop and a hard stop on every budget. Cost with no ceiling is bounded by the provider's rate limit, not by anything you designed.

Mark complete