Module 08 Summary

What this module established

A RAG pipeline is two systems, and scoring them together produces a number nobody can attribute. Retrieval hit, citation supported and answer correct are three separate columns, and the third alone will happily report success for answers that came from the model's weights and matched by luck.

Carry forward

• The eval set is built from real questions with their provenance recorded, and it must include the cases that lower the score.
• Removing a case because it fails converts the set from a specification into a description, and the score only ever goes up.
• Compare runs case by case. A change that fixes one and breaks another leaves the headline rate untouched.

Before moving on

Move on when you have two stored runs, a fixed list, a broken list, and a keep-or-revert decision that cites the broken list rather than the summary.

Mark complete