Module 06 Activity

Scenario

Your agent produces correct answers. This activity asks what it did to get them.

What you build

Trajectory scoring with mechanical checks, approval violations counted, and three independent stopping ceilings.

Steps

1. Record the tool trajectory for 20 runs. Compare answer-only scoring against trajectory scoring and report where they disagree.
2. Check the tool set and the ordering constraints separately. Assert on required precedence rather than an exact sequence.
3. Count approval violations - gated action executed with no approver recorded. Confirm the count is zero.
4. Implement step, token and time ceilings independently, and record which one fired for each stopped run.
5. Build the five-boolean trajectory scorer and confirm two people running it get identical numbers.
6. Report the ceiling-stop rate as part of your failure rate, not as a separate bucket.

Evidence to hand in

• 20 trajectories with both scoring methods and the disagreements.
• The precedence constraints and how they are asserted.
• The approval violation count.
• The three ceilings with which fired per stopped run.
• The trajectory scorer and the two-person reproducibility check.

Review checklist

• Answer-only and trajectory scoring disagree somewhere, and that case is examined.
• Precedence is asserted, not the exact sequence.
• Approval violations are a count with a threshold of zero.
• All three ceilings exist independently.
• Trajectory scoring uses no model as judge.

Mark complete