Module 08 Summary

What this module established

Cost per successful request is the honest number - retries, infrastructure and trace storage all scale with the same traffic, and a rising retry rate is a quality problem that shows up first on the bill.

Carry forward

• Key the cache on everything that changes the answer. A cache in front of retrieval bypasses the access filter unless the key carries the permissions.
• Distinguish rate limits from outages, and build a degradation ladder. Retrieval still works when the model provider does not.
• Re-run the full eval set after every optimisation, including work framed as performance. The dimension you were not watching is the one that moved.

Before moving on

Move on when your cost figure is per successful request, your cache key carries user groups, and an optimisation has been re-evaluated on all six dimensions.

Mark complete