Module 11 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. The first goal of an incident review is…
- A. Assigning blame
- B. Understanding what happened and why the existing controls did not catch it
- C. Writing a report
- D. Notifying users
2. For an LLM incident, 'what happened' must include…
- A. The error code
- B. The trace: inputs, retrieved context, and output that produced the bad result
- C. The model name
- D. The timestamp
3. The most valuable output of a review is…
- A. A summary
- B. A new eval case that would have caught it, plus the control change
- C. An apology
- D. A dashboard
4. Recurring incidents of the same type usually indicate…
- A. Bad luck
- B. A missing control rather than repeated individual mistakes
- C. A model problem
- D. User error
5. Quality drift with no code change is best explained by…
- A. Impossible
- B. Changes in the model, the corpus, or the distribution of user questions
- C. A caching bug
- D. Rate limits
---
Answer Key and Explanations
Check these only after attempting every question.
1. B - Understanding what happened and why the existing controls did not catch it
Blameless review surfaces the systemic gap; blame surfaces silence.
2. B - The trace: inputs, retrieved context, and output that produced the bad result
Without the trace the review becomes speculation about a non-reproducible event.
3. B - A new eval case that would have caught it, plus the control change
That is what converts a one-off incident into permanent coverage.
4. B - A missing control rather than repeated individual mistakes
If the same failure keeps arriving, the system permits it - that is the thing to fix.
5. B - Changes in the model, the corpus, or the distribution of user questions
Your inputs move even when your code does not - which is why continuous evaluation exists.
Practical Check
Apply this module to your own work: complete the module activity for *Incident Review and Continuous Improvement*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.