Answers and explanations · Should this version replace the last one?

Read this after attempting the practice.

1. The score hides an exchange

Baseline: overall 11/12, routine 3/4, ambiguous 4/4, boundary 4/4. Regressed candidate: overall 11/12, routine 4/4, ambiguous 4/4, boundary 3/4.

routine-04 is fixed. boundary-02 is newly failing. The average is unchanged because one gain offsets one loss numerically, but the candidate now crosses a critical staff-only boundary. Reporting the average alone conceals which behavior was lost.

2. The two gates answer different questions

Allowing one critical failure leaves new_case_failures blocking. Allowing one new failure leaves critical_route_failures blocking. The other checks still pass on the supplied regressed fixture. The evaluator's unit tests exercise these independent conditions with copied policy dictionaries.

These are logical experiments. They provide no evidence about the acceptability of a relaxed policy for actual support users. Threshold approval, expected impacts, and any release authority would need to be established separately. The supplied teaching policy remains unchanged.

3. Missing evidence is not an ordinary wrong label

The candidate must contain one output for every dataset case. Removing boundary-02 creates incomplete evidence and should produce an input error, exit code 2. An ordinary miss is a complete, valid output whose label differs from the expected label. Scoring only the remainder would reward data loss. The program rejects the mismatch before computing metrics.

4. Compare cases by identity

Reference Python solution. The first helper considers a case newly failing only when the baseline is correct and the candidate is incorrect. Sorting IDs makes output independent of input ordering. The second helper uses only the chosen slice as its denominator, while still requiring complete output evidence for the dataset.

Run the reference solution from the package root:

PYTHONDONTWRITEBYTECODE=1 python3 practice/check_exercise.py solutions/01_lab.py

All six practice checks should pass. These checks establish a small programming exercise's behavior, not a learner's readiness to operate a real AI service.

5. A correct route can accompany unacceptable content

One valid answer: an output marks the case escalate but includes someone else's private contact details in its accompanying text. The route grader sees only the label and would pass it. A separate content evaluation should check the actual response against the allowed information and have a domain reviewer validate difficult cases. The lab contains no such accompanying response text or grader.

Another valid answer: the route is correct but the handoff never reaches staff. That needs an end-to-end test of the actual handoff path, conducted with permitted test records, rather than a stronger route equality check.

6. Example decision note

candidate-regressed is blocked under juniper-local-teaching-policy-v1. This result is a replay of twelve handwritten synthetic case outputs, with invented delay and cost fields; no model or production service was evaluated. Overall accuracy remains 11/12, but the candidate fixes routine-04 while introducing a critical failure on boundary-02. It violates both the zero critical-route failure rule and the zero new-failure rule. I would inspect that boundary case and the fictional contract before preparing another candidate. Route equality does not evaluate accompanying answer content or whether a staff handoff succeeds, so even a passing fixture would not establish readiness for real users or authorize deployment.

Self-check rubric

Award one point for each independently demonstrated item:

1. Correct overall and slice fractions, with both changed IDs named.
2. Correct predictions for the two independent gate experiments.
3. Distinguishes incomplete evidence from a valid wrong route.
4. Programming helpers pass all six checks without weakening them.
5. Gives a concrete blind spot and a relevant additional evaluation.
6. Decision note states evidence type, local decision, conditions, and limits honestly.

This is a self-review guide, not an integrated grade or certificate. If item 3 or 6 is missing, revisit the lesson even if the arithmetic and code pass: trustworthy reporting is part of the task.

For the optional extension, a good response may identify a contract ambiguity rather than force a label. “Can you explain the cancellation rule for a booking I might make?” could still be general public-policy guidance, while “Cancel my booking and refund it” requests an individual action. Decide the scope with the domain owner, then version the case set. Do not silently rewrite the original test set after comparing a candidate.