Module 10 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. A release gate is…
- A. A deployment script
- B. A defined threshold the change must meet before it can ship
- C. A code review
- D. A rollback
2. Which changes should trigger the full eval suite?
- A. Only prompt changes
- B. Prompt, model, retrieval settings, corpus, and tool changes
- C. Only model changes
- D. UI changes
3. A provider silently updating a model version means you should…
- A. Do nothing
- B. Pin versions where possible and re-run evals when the version moves
- C. Switch providers
- D. Lower temperature
4. Canary or staged rollout is valuable because…
- A. It is faster
- B. Real-traffic problems surface on a small population before everyone is affected
- C. It reduces cost
- D. It avoids evals
5. Every release should have…
- A. A blog post
- B. A rollback path that can be executed quickly
- C. A new model
- D. A cost estimate
---
Answer Key and Explanations
Check these only after attempting every question.
1. B - A defined threshold the change must meet before it can ship
Without a stated threshold, 'the evals look fine' is an opinion.
2. B - Prompt, model, retrieval settings, corpus, and tool changes
All of these alter behaviour, often in non-obvious ways.
3. B - Pin versions where possible and re-run evals when the version moves
An unpinned dependency can change your product's behaviour with no deploy on your side.
4. B - Real-traffic problems surface on a small population before everyone is affected
Offline evals cannot reproduce the full diversity of production input.
5. B - A rollback path that can be executed quickly
The ability to undo is what makes shipping safe enough to do often.
Practical Check
Apply this module to your own work: complete the module activity for *Release Gates and Regression Testing*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.