Module 12 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. 'Traceable' in this capstone means a reviewer can…
- A. See the final answer
- B. Follow query, retrieval, prompt, and output for any given run
- C. Read the code
- D. See the cost
2. Your capstone must define refusal behaviour because…
- A. It is polite
- B. Without it the assistant will answer confidently from outside its sources
- C. It saves tokens
- D. Users expect it
3. The evaluation section should state…
- A. That it works
- B. The test set composition, metrics, thresholds, and current results
- C. The model name
- D. The framework version
4. Which limitation is worth stating even for a good assistant?
- A. None
- B. Corpus boundary, freshness, and the fact that citations require spot-checking
- C. Latency only
- D. The UI
5. Before any deployment, confirm that…
- A. The UI is polished
- B. Permissions are enforced at retrieval and traces do not expose data inappropriately
- C. Costs are low
- D. Streaming works
---
Answer Key and Explanations
Check these only after attempting every question.
1. B - Follow query, retrieval, prompt, and output for any given run
Traceability is what allows a claim to be challenged and a bug to be located.
2. B - Without it the assistant will answer confidently from outside its sources
Refusal is a designed capability, not an emergent one.
3. B - The test set composition, metrics, thresholds, and current results
Numbers without thresholds and set composition cannot be judged by a reviewer.
4. B - Corpus boundary, freshness, and the fact that citations require spot-checking
Stating boundaries is what produces calibrated trust rather than over-reliance.
5. B - Permissions are enforced at retrieval and traces do not expose data inappropriately
These two are the most common real incidents in shipped RAG assistants.
Practical Check
Apply this module to your own work: complete the module activity for *Capstone: Traceable RAG Assistant*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.