Module 12 Summary

What this module established

Write the scope first - the latency budget in particular constrains k, reranking and model choice for the whole build. The refusal must be a branch in code returning a flag and an empty citation list, because that is what tests can assert on.

Carry forward

• Instrument successes, not only failures. The request that most needs investigating usually completed and produced something wrong.
• Force the refusal branch with a threshold parameter, not a carefully chosen query - corpus and embedding changes break the latter.
• A single quality percentage cannot be acted on and hides which of three subsystems is weak.

Before moving on

You have finished when the suite runs with no key, the report scores three things separately against a stated budget, and a reader's questions have been recorded and answered.

Mark complete