Module 02 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. Why must correctness and grounding be measured separately?
- A. They are the same
- B. An answer can be factually right but unsupported by the provided sources, and vice versa
- C. Grounding is optional
- D. Correctness is subjective
2. Format quality deserves its own metric because…
- A. It looks nicer
- B. Downstream systems break on malformed output regardless of content quality
- C. It affects tone
- D. It is easy
3. Latency should be tracked as…
- A. An average only
- B. A distribution including p95/p99, because tail latency is what users abandon on
- C. A single number
- D. Not at all
4. Cost per successful task is more useful than cost per call because…
- A. It is smaller
- B. Retries and failed attempts are part of the true cost of getting a result
- C. It is standard
- D. It excludes tokens
5. Optimising one dimension typically…
- A. Improves all
- B. Trades against others - more retrieval raises cost and latency, tighter safety may raise refusals
- C. Has no effect
- D. Reduces cost
---
Answer Key and Explanations
Check these only after attempting every question.
1. B - An answer can be factually right but unsupported by the provided sources, and vice versa
In a sourced assistant, an unsupported correct answer is still a process failure.
2. B - Downstream systems break on malformed output regardless of content quality
Schema conformance is machine-checkable and often the first thing to regress.
3. B - A distribution including p95/p99, because tail latency is what users abandon on
Averages hide the slow tail that actually damages the experience.
4. B - Retries and failed attempts are part of the true cost of getting a result
A cheap call that fails half the time is not cheap.
5. B - Trades against others - more retrieval raises cost and latency, tighter safety may raise refusals
Naming the trade-offs prevents a 'fix' in one metric quietly damaging another.
Practical Check
Apply this module to your own work: complete the module activity for *Quality Dimensions: Correctness, Grounding, Format, Safety, Latency, Cost*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.