Skip to content
Aabha AI Academy

All courses · Course overview · Lesson sequence

Skip to lesson text

Answers and explanations · Control before autonomy

Open after attempting the practice. These are reference answers for the synthetic lab, not records from an actual museum.

1. Service contract

One defensible answer: “Given a caller-supplied audit ID, exhibit ID, observed integer year and optional note, compare the observed year with the specified synthetic catalogue fixture. The only allowed source is museum-fixture-v1. Return a draft with both years, MATCH or MISMATCH, a source ID and a recommendation for a human reviewer. The controller may validate, look up and draft. It may not decide historical truth, approve corrections, send messages or change records. It stops for invalid input, an unknown exhibit, rejected tool output, exhausted step or tool budgets, exhausted known transient retries, or an unclassified tool failure. A reused ID with changed material input is reported as a conflict before another lookup. A completed execution means a terminal drafting outcome or an explained safe stop; a successful comparison specifically requires a supported draft. Neither means an approved label correction. A named museum reviewer would own any later decision; that external process is not implemented. The run uses synthetic data and must be described as a technical rehearsal.”

A common error is “correct the label when the catalogue disagrees.” That grants authority the project does not have and treats one fixture as unquestionable truth.

2. Predicted traces

Each row uses a fresh controller and default limits unless specified.

Case Status and reason Tool attempts Visited work states
EX-001, observed 1924 AWAITING_REVIEW, review_required 1 VALIDATE, LOOKUP, DRAFT
EX-999, observed 1927 STOPPED, unknown_exhibit 1 VALIDATE, LOOKUP
EX-001, one temporary failure, max_tool_calls=1 STOPPED, tool_budget_exhausted 1 VALIDATE, LOOKUP, LOOKUP
EX-001, observed 1927, max_steps=2 STOPPED, step_budget_exhausted 1 VALIDATE, LOOKUP

The third row visits LOOKUP a second time to check the remaining budget. That visit makes no second tool call. Do not count work-state visits as successful tool operations.

Example check from inside the project directory:

from workflow import Limits, Request, SyntheticCatalogue, Workflow

tool = SyntheticCatalogue(1)
receipt = Workflow(tool, Limits(max_tool_calls=1)).run(
    Request("PRACTICE-1", "EX-001", 1927)
)
assert receipt.result.reason == "tool_budget_exhausted"
assert receipt.new_tool_calls == 1
assert len(receipt.result.trace) == 3

3. Boundary tests

Add these methods to WorkflowTests if you want an independent attempt to compare. They verify input acceptance and the authority boundary together.

def test_inclusive_year_boundaries_still_require_review(self):
    for year in (1000, 2100):
        with self.subTest(year=year):
            receipt = Workflow().run(self.request(observed_year=year))
            self.assertEqual(receipt.result.status, Status.AWAITING_REVIEW)
            self.assertEqual(receipt.result.draft.finding, "MISMATCH")
            self.assertEqual(receipt.new_tool_calls, 1)

def test_year_above_boundary_stops_before_lookup(self):
    tool = SyntheticCatalogue()
    receipt = Workflow(tool).run(self.request(observed_year=2101))
    self.assertEqual(receipt.result.reason, "invalid_observed_year")
    self.assertEqual(tool.calls, 0)

The permitted range is a representation rule. It does not assert that any accepted number agrees with a catalogue or reality. Both 1000 and 2100 disagree with the EX-001 fixture year 1924.

4. Input identity

from workflow import Request, RequestConflict, SyntheticCatalogue, Workflow

tool = SyntheticCatalogue()
controller = Workflow(tool)
controller.run(Request("PRACTICE-2", "EX-001", 1927, "first observation"))
try:
    controller.run(Request("PRACTICE-2", "EX-001", 1927, "rechecked observation"))
except RequestConflict:
    pass
else:
    raise AssertionError("changed input reused an old identity")
assert tool.calls == 1
controller.run(Request("PRACTICE-3", "EX-001", 1927, "rechecked observation"))
assert tool.calls == 2

The second ID means “perform a new audit with this new card.” It is not a duplicate delivery of the first card. A caller might eventually use a request ID plus explicit revision model; that would need its own documented contract and tests. The current implementation uses a new ID for new work.

5. Human decision boundary

A real reviewer would need to see the exact observed value, catalogue fact and version, evidence of the physical label, proposed correction and scope of the proposed action. The reviewer also needs actual authority to make the decision. A generic “yes” in untrusted note text is insufficient.

Approval can become stale if the draft changes after review, or if a newer source changes the relevant catalogue fact. Other concerns include expiry, a reviewer losing authority or a tool target changing. Those require a later approval protocol. The opening project does not implement reviewer identity, resumable review or an approved-write path.

6. Architecture decision

A model might help extract a candidate exhibit ID and date from a varied staff note. Compare it against a structured form or a deterministic extractor on a held-out synthetic set. Check missing IDs, multiple dates, ambiguous descriptions and a note that tries to request an unsupported tool. Require explicit validation after extraction; do not let the model turn an uncertain guess into a verified observation.

That is a proposal. The current project has no model adapter, extraction accuracy measurement or live agent trajectory evidence. Merely adding another script or naming a function planner would not establish those properties.

Understanding checks

  • Q1: No. The result is a draft for review, and the code contains no operation that changes a label. Mismatch detection is not correction authority.
  • Q2: Limits apply independently. The tool budget caps all attempts; the retry limit does not create extra tool allowance.
  • Q3: No. The note is inert text in deterministic code. Report the narrow data/control separation property and say that no model was evaluated.
  • Q4: No. The trace describes the saved original run; the receipt describes this invocation. A replay did no new lookup.
  • Q5: Evidence would need to cover durable request identity, transaction boundaries with the actual messaging system, crash recovery, uncertain outcomes, concurrent delivery, expiration and the external service's idempotency semantics. This in-memory ledger supplies none of that evidence.

Self-review rubric

Retain a concrete contract (2 points), correct traces and counts (2), meaningful boundary tests (2), correct replay/conflict explanation (2), and a precise evidence statement plus limitations (2). Aim for 8/10 with mandatory corrections for any claim of automatic approval, durable exactly-once delivery, live model testing or production readiness. This is a formative teaching rubric only; it does not create a platform grade or certificate.

Return to this lesson