Module 03 Activity

Scenario

Your graph has model calls in it. Some of them are doing work an if statement could do, and they are costing you money and reliability.

What you build

A cost and reliability audit of every step, with at least one model step moved into code and one made checkable.

Steps

1. List every step and mark it code or model. For each model step, try to write the rule down; if you can, it belongs in code.
2. Move at least one decision from the model into code, and measure the before-and-after latency of that step.
3. For a model step that must stay, write a validator that checks parse, shape and allowed values separately.
4. Feed it three deliberately malformed outputs, including one that is valid English wrapping valid JSON.
5. Add a deterministic fallback and a decided_by field, then report the fallback rate over twenty runs.
6. Price the whole workflow over a thousand runs, splitting cost and latency by code versus model.

Evidence to hand in

• The step inventory with code/model and the written rule where one exists.
• The validator and its results on three malformed outputs.
• The fallback, the decided_by field, and the measured fallback rate.
• The thousand-run cost and latency table.

Review checklist

• At least one decision moved from model to code, with a measurement.
• The validator checks allowed values as data, not as a prompt instruction.
• One test case is valid English containing valid JSON.
• decided_by is recorded, and the fallback rate is a number.
• The cost table shows what share of cost and latency the model steps carry.

Mark complete