Module 10 Summary

What this module established

Read traces backwards from the symptom, asking whether each step was right given its input. Steps downstream of a bug behave correctly on bad data, which makes them look guilty - and the model is nearly always both the slowest step and the first suspect.

Carry forward

• Record every input including prompt, policy and chain versions. A difference you cannot attribute reads as model non-determinism.
• Score retrieval and generation separately. One accuracy number hides which of two systems is failing.
• Split trace retention by field. A single policy forces you to over-retain the sensitive part.

Before moving on

Move on when you have found a planted fault by reading backwards, attributed a two-run difference to a config field, and set retention per field.

Mark complete