Module 11 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. A complete design brief must state the corpus boundary because…
- A. It looks thorough
- B. It defines what the assistant can and cannot be expected to answer
- C. It reduces cost
- D. It speeds retrieval
2. Your evaluation plan should specify…
- A. Only the number of questions
- B. The question mix, the metrics, the pass thresholds, and when the evals are re-run
- C. The UI colours
- D. The embedding dimensions
3. The brief should name a refusal policy because…
- A. It is optional
- B. Deciding in advance what the assistant will not answer prevents confident invention later
- C. It reduces latency
- D. Users expect it
4. Which risk belongs in the brief even for an internal tool?
- A. None
- B. Permission leakage, stale documents, and unverified citations reaching a decision
- C. Server cost
- D. Font choice
5. The honest closing statement of a good brief includes…
- A. A performance guarantee
- B. The known limitations and what has not yet been tested
- C. A launch date only
- D. A marketing summary
---
Answer Key and Explanations
Check these only after attempting every question.
1. B - It defines what the assistant can and cannot be expected to answer
Unstated scope is how assistants get blamed for questions they were never designed to handle.
2. B - The question mix, the metrics, the pass thresholds, and when the evals are re-run
An eval plan without thresholds and a re-run trigger is a one-off demo, not a quality gate.
3. B - Deciding in advance what the assistant will not answer prevents confident invention later
Refusal behaviour is a design decision, not an emergent property.
4. B - Permission leakage, stale documents, and unverified citations reaching a decision
Internal does not mean low-consequence; these three cause most real incidents.
5. B - The known limitations and what has not yet been tested
Naming what you have not verified is what allows a reviewer to judge readiness.
Practical Check
Apply this module to your own work: complete the module activity for *Capstone: RAG Design Brief and Evaluation Plan*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.