Module 10 Activity

Scenario

Everything measured so far has to become something that blocks a bad release automatically.

What you build

A gate set running in CI, a scheduled eval catching provider-side drift, per-case version comparison, and a rollback path that includes configuration.

Steps

1. Define every gate with a metric, bar, comparison and severity. Include newly-broken-cases at zero.
2. Use warnings deliberately so blockers keep their force, and justify each warning.
3. Schedule the eval set to run independently of your deploys, and explain what that catches.
4. Compare two versions per case and produce the fixed and broken lists. Make the release decision from the broken list.
5. Make prompts and retrieval settings roll back with the code. Prove a rollback restores both.
6. Implement an override that requires a recorded reason, and set a monthly override review.

Evidence to hand in

• The gate set with all four components per gate.
• The scheduled eval and what it caught, or would catch.
• A per-case version comparison with both lists.
• Evidence a rollback restored code AND configuration.
• The override mechanism and the review schedule.

Review checklist

• Every gate has a metric, bar, comparison and severity.
• newly-broken-cases == 0 is a blocker.
• The eval runs on a schedule, not only on deploys.
• The release decision cites the broken list, not the total.
• Rollback restores the prompt and policy versions, not just the binary.
• Overrides are possible and recorded.

Mark complete