Module 02 Activity

Scenario

Your system reports one quality number. This activity replaces it with six that can each fail specifically.

What you build

Six dimensions measured separately against six bars, with a costed configuration decision.

Steps

1. Define bars for correctness, grounding, format, safety, latency and cost, noting which direction each is better in.
2. Define correctness as facts present or absent, and grade four answers including one paraphrase and one that drops a qualifier.
3. Implement the three-state grounding check: grounded, uncited, unsupported. Report how many of your answers are uncited-but-correct.
4. Add a format check that runs on every production response, not just the eval set.
5. Price two configurations with their measured correctness, and state what the last point of quality costs.
6. Report all six dimensions and identify which one would have been hidden by an average.

Evidence to hand in

• The six bars with their directions.
• Four graded answers with the facts checked.
• The three-state grounding results with the uncited-but-correct count.
• The format check running in production.
• The costed configuration comparison including correctness.
• The six-dimension report and the dimension an average would hide.

Review checklist

• Higher-is-better and lower-is-better metrics are not mixed into one score.
• Correctness is graded on facts, so a paraphrase scores as correct.
• Grounding has three states, not two.
• Format is checked on every response.
• The configuration decision cites measured correctness, not an assumption.

Mark complete