All courses · Course overview · Lesson sequence

Capstone: prepare a defensible local release recommendation

Assignment and supplied system

You are the fictional owner of Juniper Workshop's support router. A teammate recommends the candidate because its four visible teaching holdout route labels pass. Review the other evidence before responding. Your work is entirely offline, uses synthetic inputs, and performs no deployment or real model inference.

Read module 10 and the earlier modules first. The collector is opslab.capstone_evidence(opslab.load()); the learner driver is starter.py. It reuses the actual route evaluator, dataset audit, reviewer/calibration mechanics, repeated comparison, window monitor, drift calculation, stage simulation, and maintenance review. Its manifest covers the required original input/code files. The driver also records a digest of the submitted decision code when implemented.

Deliverables

Implement blocking_reasons(evidence) and recommend(evidence) in the starter. The first returns sorted strings; the second returns decision, blocking_reasons, recovery_required, and live_authorization. Do not mutate inputs. Missing required evidence must block. live_authorization is always False.
Write a 140–220 word decision note naming evidence type, exact important case/window IDs, local recommendation, limits, and an owned next step.
Propose two original synthetic cases that challenge different failure mechanisms. Give unit/context, group, split, expected route, content criterion, rationale, and review plan before assigning any candidate output. Keep them in a proposed new version; do not rewrite the supplied evidence to make the current candidate pass.
Write a short paper recovery/maintenance note: known baseline artifact, compatibility check, verification after recovery, policy/provider review owners, and what real permission/data would still be needed. Do not execute any of it against a live system.

Decision contract

Required top-level sections are evidence_kind, dataset, route, review, calibration, comparison, monitor, rollout, maintenance, drift, manifest_changed, rubric_review_complete, and follow_up_reviewed. Missing nested fields used by the rules also block instead of crashing or assuming a pass: section decisions/states; complete boolean window flags; calibration rates and nonnegative integer FP; maintenance timing/kind; drift distance/threshold/review flag; and the changed-artifact list. Validate the evidence shape before applying comparisons. JSON strings such as "false" are not boolean flags.

Apply each condition independently:

Condition	Requirement for a local simulation pass
Evidence kind	Exactly `synthetic_fixture_replay`; the exercise cannot authorize other evidence modes.
Dataset, route, semantic review, paired comparison	Each section's `decision` must be `PASS`.
Proposed judge calibration	`accept_precision` and `false_acceptance_rate` must be defined; FP must be zero in the supplied calibration set. This is a teaching rule, not a general judge-quality standard.
Windows	At least one supplied window; no `operational_alert` or `quality_review_required` flag.
Stage simulation	State must be `COMPLETE_SIMULATION`, not HOLD or RECOVER.
Maintenance	No open returned item of kind `policy` or `provider` with timing `due` or `overdue`.
Mix shift	If distance exceeds the supplied threshold, `reviewed` must be explicitly `True`. A review flag needs supporting explanation; distance is not accuracy.
Artifact identity	`manifest_changed` is empty.
Review completeness	Both rubric review and incident follow-up are explicitly reviewed.

If any condition fails, return BLOCK; otherwise return PASS_SIMULATION. Set recovery_required only when the supplied rollout state is RECOVER, and keep it false for a mere HOLD. Missing evidence never turns into an automatic pass. Reason string names are your choice, but they must be specific and sorted.

The collector's rubric/follow-up/mix-review flags are false because those teaching reviews have not been completed. Changing a flag in a test fixture demonstrates logic, not genuine review or system repair.

Run and inspect

From the package root:

python3 -B capstone/starter.py
python3 -B capstone/check_submission.py capstone/starter.py

The unimplemented starter exits 3 with starter_incomplete; the checker fails until the helpers are implemented. Its eleven checks exercise a passing route with other failures, independent quality flags, undefined calibration, missing evidence, due reviews, drift review, immutability, and an explicitly synthetic all-clear packet. They are technical exercise checks, not a learner pilot or platform grade.

After implementing the functions, run the driver and inspect the full returned evidence. A completed driver returns exit 0 when computation succeeds even if the recommendation is BLOCK. Inspect the recommendation, not just the shell exit code. The module 1 gate.py has a different documented exit convention; do not assume every tool treats a block as a process failure.

Do not pipe a report over an input file. No package download, account, credential, network connection, or running service is required. Reference explanation and reference code remain separate.

Self-review rubric: 20 points

Area	Points	Evidence required
Cross-layer decision	5	All eleven checker tests pass; missing evidence and each independent blocker are handled without mutating inputs.
Metric interpretation	4	Correct route scope, reviewer/calibration denominators, four distinct cases vs twelve repeats, and operational union vs independent critical flag.
Proposed cases	4	Two original cases with distinct mechanisms, explicit units/groups/labels/rationales, content checks, and a review/version plan.
Decision and recovery	4	Concrete blockers and IDs; prioritized owned next steps; known recovery artifact and verification; unresolved policy/provider work retained.
Honesty of claims	3	Explicit synthetic provenance; no claim of real model performance, production verification, learner pilot, publication, certification, or execution authority.

Self-review passes at 16/20 and all eleven technical checks pass and no false real-world claim is made. This is an original classroom rubric, not an Aabha-integrated assessment or certificate. A score from copying the reference without explaining its limits does not demonstrate the intended capability.

Download this source file