Unit 09.03: Setting a hard budget per run

One ceiling is not enough, because a run can be cheap and endless or fast and expensive.

Tokens, money, time - checked after every step

Three ceilings, evaluated after each step, with a hard stop on the first breach.

The code runs a budget through seven steps.

class Budget:
    def __init__(self, max_tokens, max_usd, max_seconds):
        self.max_tokens, self.max_usd, self.max_seconds = max_tokens, max_usd, max_seconds
        self.tokens = self.usd = self.seconds = 0.0

    def charge(self, tokens, usd, seconds):
        self.tokens += tokens
        self.usd += usd
        self.seconds += seconds
        for name, used, cap in [("tokens", self.tokens, self.max_tokens),
                                ("usd", self.usd, self.max_usd),
                                ("seconds", self.seconds, self.max_seconds)]:
            if used > cap:
                return False, f"{name} budget exceeded: {used:.3f} > {cap}"
        return True, ""


budget = Budget(max_tokens=15_000, max_usd=0.05, max_seconds=60)
for step in range(1, 8):
    ok, reason = budget.charge(3_000, 0.009, 9)
    print(f"step {step}: tokens={budget.tokens:>6,} usd=${budget.usd:.3f} "
          f"{'ok' if ok else 'STOP -- ' + reason}")
    if not ok:
        break

# Three ceilings, because a run can be cheap and endless, or fast and expensive.
# Check after every step and stop the run -- a budget that only reports at the
# end is a report, not a control.

The run stops at step six, on the token ceiling, before the dollar ceiling would have caught it. Which ceiling fires first depends on the workload, which is exactly why all three exist.

The check is after *every* step. A budget evaluated only at the end is a report - it tells you what you spent, which you would have discovered from the invoice anyway.

The mistake this prevents

The mistake is setting the budget from the happy path. A budget sized for a normal run leaves no headroom for a legitimate retry, so you spend the next month raising it. Size it from the worst legitimate run and let the loop detectors catch the pathological ones.

Takeaway

Set ceilings on tokens, money and time, check them after every step, and stop the run on the first breach. Size them from the worst legitimate run, not the typical one.

Mark complete