Control before autonomy

At the fictional North Quay Museum, an exhibit label says 1927. The catalogue card says 1924. A colleague asks for “an agent that fixes wrong labels.” It is tempting to start with a model prompt: read the label, look up the catalogue and make the correction.

Before choosing a model, pause over the word fixes. A catalogue card can be wrong too. Someone must decide whether this is a transcription error, a disputed date or a stale source. A useful first tool can prepare a comparison for that person. It should not quietly turn a mismatch into a changed exhibit label.

We will build that smaller tool. By the end, you will be able to explain who chooses the next step, trace a bounded execution, test a stop condition and state exactly what your evidence proves. You will run a program that makes no model calls. It is the controllable baseline against which a later agent can be judged.

Turn the request into a job we can check

Start with one request card:

request_id: AUDIT-001
exhibit_id: EX-001
observed_year: 1927
note: The label beside the west case uses this year.

The request ID names this unit of work. The exhibit ID names the thing we will look up. The observed year is a claim someone has recorded; the program has not seen the physical label. The note supplies context but grants no authority. These roles are different even though Python represents several of them as strings.

Our output contract is also small: produce the two years, a finding, a source ID and a recommendation for review. If we cannot produce that record safely, return a stop reason. The program may read the supplied synthetic catalogue and compare integers. It may not update a catalogue, change an exhibit label, notify a colleague or approve its own recommendation.

That contract gives us something observable. “Make the museum accurate” is too broad to test with two fixture facts. “For EX-001 and observed year 1927, draft a mismatch against catalogue year 1924 and stop for review” can be checked.

Who chooses what happens next?

Imagine two designs. In the first, your code always validates the card, looks up the exhibit and drafts the comparison. In the second, a language model can decide whether to search a catalogue, ask for another observation or finish with a draft. These can look similar from the outside. The important difference is who selects the path. Anthropic's engineering guide distinguishes predefined workflows from agents whose models direct their next steps and tool use.

Our opening project is the first design: a deterministic orchestrated workflow, with no LLM at all. Given the same request, fixture, limits and initial failure count, it behaves the same way. A hard-coded retry is still a workflow. A file named agent.py or a loop does not make a model-directed agent.

In Module 4, a deterministic planner proposes one action from a short list. The controller decides whether that action is permitted, whether its arguments are valid and whether any budget remains. The planner's flexibility must not become permission to invent a new tool. A future live-model adapter is not implemented in this course.

For this particular comparison, fixed logic is sufficient. A model might become useful if staff submit varied natural-language descriptions that require interpretation. That is a new hypothesis to evaluate, not a reason to put integer comparison under model control.

Read the controller as a sequence of rooms

Open the visual. There are three work states: VALIDATE, LOOKUP and DRAFT. State means the part of the job currently being performed. It is not a mood or a natural-language progress message.

If the diagram extends beyond the screen, scroll inside it or read the text version below.

Read the diagram as text

• Control the path before adding a planner
• Original Aabha teaching visual · all tools and facts in this exercise are synthetic
• Inside the deterministic rehearsal
• Controller-enforced limits
• Default: 6 work-state visits · 2 tool attempts · 2 retry-sequence attempts total
• 1 VALIDATE
• Check card shape
• and year range
• 2 LOOKUP
• Read fixture fact
• and validate output
• 3 DRAFT
• Compare years
• and cite source
• Retry only a known temporary read failure
• Each attempt consumes budget
• STOPPED
• Invalid input or tool output · absent exhibit
• Unclassified failure · exhausted limits
• AWAITING_REVIEW
• Match or mismatch draft
• No automatic approval
• Human boundary
• Real source review
• Correction decision
• Any actual change
• Outside this project
• No implementation
• or authorization implied
• Replay is a separate question
• Same ID + same input → saved terminal result, zero new tool calls. Changed input + same ID → conflict.
• Ledger exists only in this controller object. It does not prove durable or concurrent exactly-once behavior.

Validation checks shape and range. An exhibit ID shaped like EX-999 is valid input even though it is absent from the catalogue. A Boolean such as True is invalid as a year; Python's type hierarchy would otherwise let it behave like the number one. Input validation prevents confusion about representation. It does not prove that the observed year is true.

Lookup calls one allowed tool, SyntheticCatalogue.lookup. That tool reads two fictional facts stored in code. Its output must identify the requested exhibit, carry an integer year in range and have the expected source reference. An answer about EX-002 is not acceptable evidence for EX-001 merely because it is well formed.

Drafting compares the accepted numbers and prepares a record. Both a match and a mismatch finish at AWAITING_REVIEW. A matching pair does not prove that the original observation or catalogue is correct. It only means these two supplied values agree.

The other terminal outcome is STOPPED. A terminal outcome ends this execution. A stop is useful information: no draft was produced because an input, tool, limit or assumption failed. It should not be counted as a completed comparison.

In plain text, the visual says: validate the card, make a bounded lookup, prepare a draft, and stop before any approval or real change. Known temporary read failures may revisit lookup. Invalid data, missing facts and exhausted limits lead to a stop.

Follow one complete run

From the course folder, run:

PYTHONDONTWRITEBYTECODE=1 python3 project/demo.py

The header identifies the output as a synthetic deterministic rehearsal. The first case uses AUDIT-001, EX-001 and 1927. The fixture says EX-001 has catalogue year 1924 and source ID CAT-v1:EX-001.

The finding is MISMATCH, and the recommendation asks for source review before any label change. This is a complete drafting run, but it is not an approved correction. No approval operation exists in the program.

Read Workflow._execute in workflow.py. A fixed for loop bounds work-state visits. Each branch names the next state or returns a terminal result. There is no while True, hidden remote worker or instruction that tells the tool to keep trying until it succeeds.

The trace records visited work states, not a hidden chain of thought. A trace entry says what the software did, such as “catalogue fact accepted.” It is deliberately concise. It does not need raw note text or sensitive exception details to establish the path.

Make failure part of the normal design

Now inspect the case labelled “one temporary failure.” The synthetic tool is configured to fail once before returning a fact. The trace becomes VALIDATE → LOOKUP → LOOKUP → DRAFT. There are four work-state visits and two tool calls. The failed attempt consumed budget even though it produced no fact.

Three limits do different jobs. max_steps bounds total work-state visits. max_tool_calls bounds tool attempts. max_attempts bounds attempts in this single-tool retry sequence. The defaults are six, two and two. Retry count is not “two extra attempts”; it is a maximum of two attempts in total. A lower bound can end the run first.

Suppose max_steps=2. Validation takes one visit and the successful lookup takes the second. There is no third visit left to draft the comparison. The result is STOPPED, reason step_budget_exhausted, even though the lookup succeeded. That is correct behavior under this budget. Raising a limit is a design change to justify, not something a request note can demand.

We retry only a known synthetic temporary read failure. An absent exhibit is not temporary: repeating the same lookup adds no new evidence. An unexpected exception is unclassified, so the controller stops instead of guessing that retry is safe.

Imagine a different future tool that changes a record. If the connection drops after the service receives the command, you may not know whether the change happened. The right answer is not automatically “retry twice.” You need the service's outcome/reconciliation and idempotency contract. That future write problem is outside this read-only lab. AWS's idempotency article explains why request identity and atomic handling matter when repeated calls can create side effects.

Distinguish a replay from another execution

The demo sends the same request to the same Workflow object twice. The second receipt says replayed=True and new_tool_calls=0. The original result and trace are returned from an in-memory ledger.

The ledger pairs the caller's request ID with a fingerprint of the material input: exhibit, year, note, schema version and fixture version. The fingerprint helps detect changed content; it is not a replacement for the caller's intent. Two different IDs can express two separate audits of the same exhibit. Reusing AUDIT-001 with a changed observed year is a conflict, not a silent correction.

This is a narrow rehearsal of idempotent replay. Close the process or create a new controller and its ledger is gone. There is no concurrent access control, durable checkpoint or external transaction. The program does not claim exactly-once execution. Even a stopped terminal result is cached in the same object; a deliberate new attempt needs a new request ID.

The receipt separates new work from old evidence. Its new_tool_calls describes this invocation. The saved result's tool_calls and trace describe the original run. Reporting the old trace as a second successful lookup would misstate what happened.

Treat text as data, and state the limit of the demonstration

The last demo note says to ignore the rules, approve the record and send a message. It has no effect on routing. The code never interprets note text as an action or tool name, and the available tool cannot send anything. This shows that inert data cannot choose an operation through this particular interface.

It does not show that an LLM resisted prompt injection: no LLM read the note. When a future planner consumes natural language, adversarial cases need their own evaluation. OpenAI's agent safety guidance discusses structured data boundaries, tool confirmation and the remaining risks of untrusted input. A constrained interface helps, but a schema cannot prove a claim true or establish authority by itself.

Practice — keep your evidence

Attempt these tasks before opening the separate solutions. Use only fictional values. Save your contract, table, test changes and a short evidence statement.

1. Write the service contract. In 150–250 words, name the input, allowed source, output, authority boundary and at least three stop conditions. Explain what “complete” means for this module.
2. Predict before running. For each case, predict status, reason, tool calls and visited states: EX-001 observed as 1924; EX-999 observed as 1927; EX-001 observed as 1927 with one temporary failure and max_tool_calls=1; the same mismatch with max_steps=2. Run a small Python snippet or adapt a test to check your predictions.
3. Add meaningful boundary checks. Add tests for observed years 1000 and 2100, and for 2101. The first two pass input validation and still require review; the last stops before a lookup. Explain why the fixture can legitimately disagree with either accepted boundary year.
4. Change input identity deliberately. Run a request, then change only its note while keeping its ID. Check the conflict and show that the tool's call count did not rise. Give the changed request a new ID and explain what new work that identity expresses. Do not remove conflict detection to make the test pass.
5. Investigate a boundary that is missing. In a scratch explanation, describe what a reviewer would need to approve an actual correction: the exact draft, source version, proposed action and authority. Name two ways approval could become stale. Do not add a real write or approval operation.
6. Make a justified architecture decision. A colleague asks for a model agent in this opening project. State one task where a model could add useful interpretation, one baseline to compare it against and one new failure case you would test. Distinguish planned work from working code.

Check your understanding

Q1. A run finds a mismatch and ends at AWAITING_REVIEW. Did it fix the label? Which evidence supports your answer?

Q2. A temporary failure uses the last permitted tool call. Why can the next lookup be blocked even though another retry is allowed by max_attempts?

Q3. The malicious-looking note has no effect. Can you report that the project passed an LLM prompt-injection test? Explain precisely.

Q4. A replay returns a trace containing one lookup, but the receipt reports zero new calls. Is that contradictory?

Q5. Which facts would you need before claiming that duplicate real messages are prevented across process restarts?

Before you continue

Run the supplied tests:

PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s project -v

You should be able to explain a passing test and an untested risk in your own words. A useful evidence statement is: “On Python 3.14.4, the supplied synthetic checks passed for the local deterministic controller. I have not evaluated a live model, a real museum process, durable replay or learner experience.” Replace the version and outcomes with what you actually observed.

Continue to Module 2 with your contract, trace and evidence statement. Later modules expand the controller and run an actual framework while retaining synthetic tools and planners. This local edition does not provide platform grades or certificates.