All courses · Course overview · Lesson sequence

Defend a decision across the evidence layers

The new router gets every held-out route label right. Its dashboard could show a reassuring green square. But the response reviewers found unsupported approval and disclosure text, repeated trials lost quality, and the paper rollout reached a recovery condition. Which result should the owner trust?

The answer is not to choose the most convenient report. Each layer answers a different question. This module brings them together in a reviewable local recommendation, with an executable starter, a separate reference solution, and a self-review rubric. All inputs remain synthetic. The capstone does not publish a course, change a service, operate an account, or certify production readiness.

Follow the identities into the decision

Run from the package root:

python3 -B capstone/starter.py

The driver collects the available teaching evidence and prints starter_incomplete, the synthetic evidence type, and a passing holdout route decision. It exits with code 3 because two decision functions are intentionally unfinished. That is a runnable starting point, not a hidden completed solution. Implement the helpers using the full brief before comparing with the reference.

The collector derives a four-case holdout route dataset from the reviewed register and applies the original gate. The baseline and candidate both match all four labels. Because this teaching holdout is visible, the result is not an unseen real benchmark. It is a clean example of a true statement with a narrow scope: the supplied labels pass route equality.

The collector also retains the eight-file manifest, rubric summary, judge calibration, paired comparison, window summaries, drift, paper rollout, maintenance state, and fictional incident follow-up. These are not interchangeable votes. A critical condition should not be outvoted by several unrelated green checks.

A passing route label does not approve the whole application — Five evidence cards: route gate passes four of four labels; semantic review flags s2 and s3; repeated trials block at ten of twelve and p95 900 milliseconds; window w3 triggers recovery; pending policy, provider, rubric and follow-up review block. Combined local recommendation is BLOCK. Evidence is synthetic and no live action is authorized.

Text alternative: route labels pass; semantic review fails; repeated comparison blocks; operational/recovery evidence blocks; pending review blocks. The combined local recommendation is BLOCK and recovery is required in the fictional simulation. Live authorization remains false.

Work through the supplied scenario

Route equality passes. Semantic review returns REVIEW for s2 and s3, including a disagreement and a false consensus that still need domain attention. Proposed judge calibration has one false acceptance among three reference-unacceptable cases. The paired comparison has ten correct observations out of twelve and a 900 ms nearest-rank p95, so it blocks. The windows contain both operational alerts and a critical quality flag. The stage simulation returns RECOVER. A policy review is overdue, a provider review is due, and the changed input mix has not been reviewed.

The combined result should therefore be BLOCK, with recovery_required: true and live_authorization: false. “Recovery required” is a recommendation inside the fictional paper simulation, not an executed recovery or permission to operate Aabha. The reference program prints the evidence and a digest of its own decision code so the specific calculation can be inspected.

The offline prerequisite in the rollout fixture is True, even though this broader evidence packet contains blockers. That is deliberate: module 7 isolated stage mechanics using a supplied flag. The capstone must expose this inconsistency rather than treating a manually supplied boolean as a complete release approval. A real orchestrator would connect validated prerequisite evidence before exposure, not merely trust an optimistic flag.

Define what passing would require

For this exercise, a pass is named PASS_SIMULATION. The brief requires complete evidence, a passing dataset audit and route gate, resolved passing semantic review, a calibrated proposed judge with no false acceptances in the supplied set and defined denominators, a passing paired comparison, healthy windows, a completed stage simulation, no unresolved due policy/provider review, reviewed significant mix shift, unchanged manifest, and reviewed rubric/incident follow-up.

These are toy policy requirements, not universal thresholds. They also do not mean that setting the flags to true constitutes review. The self-review requires corresponding evidence and an explanation. The checker's all-clear fixture is constructed to test decision logic; it is not a repaired model or an empirical success story.

Some investigation can continue while a recommendation blocks. You can clarify rubric anchors, propose independently written cases, identify affected policy versions, and improve the paper recovery plan. You should not infer permission to deploy, collect real messages, or alter a live schedule from progress on those local tasks.

Make the next action concrete

A useful final note names the candidate and evidence type, explains the strongest blockers, cites case/window IDs, states what is known and unknown, and proposes an owned next step. Avoid an unprioritized list of every number. For this scenario, address the critical approval/privacy path and the recovery condition before chasing a smaller average cost improvement.

The NIST Generative AI Profile provides broader lifecycle context for trustworthiness work. Our capstone is a bounded rehearsal of evidence reasoning, not certification against that profile. Its self-review rubric is separate from platform grading, learner identity, completion records, and certificates.

Practice: complete the capstone

Use BRIEF.md in order: inspect the packet; implement the two helpers; run the checker; design two original synthetic additions before assigning outputs; and write a decision plus a recovery/maintenance note. Keep solutions separate until you have an attempt.

When ready, compare with the explained reference. A correct capstone may end in a well-supported block. Learning to preserve the boundary between a local score, a release decision, and execution authority is part of operating an AI application responsibly.

Read separate answers and explanations