Module 01 Summary

What this module established

The same prompt produces different correct answers, so design tests, caches and comparisons around variation rather than trying to suppress it. Confine the non-determinism to one step and validate immediately after - everything past that point is ordinary testable software.

Carry forward

• Calls cost money whether or not they produce an answer. Measure cost per successful result, and put an idempotency key behind anything a timeout could have half-completed.
• A hosted model can change behaviour with no commit on your side, which is why evals run on a schedule rather than only on your pushes.
• Write the must-never list first, with an enforcement per line that is not a prompt instruction.

Before moving on

Move on when your pipeline has one non-deterministic step, validation immediately after it, and two prohibitions that are now tests.

Mark complete