Module 08 Activity

Scenario

The system is correct and the bill is rising. This activity finds where the money and the milliseconds actually go.

What you build

Honest cost per successful request, per-span latency, a safe cache key, a degradation ladder, and a re-evaluated optimisation.

Steps

1. Measure cost per SUCCESSFUL request including retries, infrastructure and trace storage. Report the retry overhead as a percentage.
2. Break latency down per span and identify the lines you can actually remove.
3. Design a cache key containing everything that changes the answer, including policy version and user groups. Prove two users get different keys.
4. Distinguish rate limits from outages in your retry logic, and build the degradation ladder including the retrieval-only rung.
5. Make one optimisation, then re-run the FULL eval set and report every dimension that moved.
6. State whether you kept the optimisation and why.

Evidence to hand in

• Cost per successful request with the retry overhead.
• Per-span latency with the removable lines identified.
• The cache key and the two-user proof.
• The retry logic and the degradation ladder.
• Full eval results before and after the optimisation.
• The keep-or-revert decision.

Review checklist

• Cost is per successful request and includes retries.
• The cache key includes user groups, so a hit cannot bypass access control.
• The degradation ladder includes returning retrieved passages without a generated answer.
• The optimisation was re-evaluated on every dimension, not just the one being optimised.
• A dimension other than the target moved, and that is reported.

Mark complete