Unit 10.02: Writing the stopping rules and budgets

Five ways to stop, and only one of them is success.

Goal, steps, time, actions, repeats - plus budgets

Each ceiling bounds a different thing, and the outcome records which one fired.

Below is the full set.

{
  "stopping_rules": {
    "goal_met": "the decision and its evidence are produced",
    "max_steps": 8,
    "max_minutes": 5,
    "max_actions": 1,
    "repeat_ceiling": "same tool with same arguments 3 times"
  },
  "budgets": {
    "max_tokens": 12000,
    "max_usd": 0.04
  },
  "on_ceiling": "stop, record which ceiling, escalate with the trail so far",
  "outcomes_recorded": [
    "success",
    "escalate",
    "failure",
    "stopped_by_ceiling"
  ]
}

five independent ways to stop, and only one of them is success

max_actions: 1 is the important line. This workflow posts at most one credit per run, so a second attempted action is a bug regardless of what the step and token ceilings say - and neither of those would catch it.

on_ceiling specifies escalation with the trail so far, rather than a bare failure. A run that stopped at a ceiling has usually done useful work, and handing that to a person is better than discarding it.

The mistake this prevents

The mistake is setting ceilings from the typical run. Size them from the worst legitimate run, or you spend the next month raising them and eventually stop trusting them.

Takeaway

Bound goal, steps, time, actions and repeats separately, record which fired, and escalate with the trail. Size ceilings from the worst legitimate run.

Mark complete