Module 01 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. Why can't traditional software reliability practice be applied unchanged?
- A. LLM apps are slower
- B. The same input can produce different outputs, so pass/fail assertions alone are insufficient
- C. They use more memory
- D. They have no tests
2. 'It worked in the demo' is weak evidence because…
- A. Demos are short
- B. A handful of favourable examples says nothing about failure rate on real traffic
- C. Demos use small models
- D. Users differ
3. Reliability targets should be set…
- A. As high as possible everywhere
- B. Per use case, based on the cost of a wrong answer
- C. By the model vendor
- D. At 100%
4. The most under-measured failure in LLM apps is usually…
- A. Latency
- B. Confidently wrong output that no one flags
- C. Downtime
- D. Cost
5. LLMOps differs from MLOps mainly because…
- A. It is newer
- B. Behaviour is driven by prompts, context, and third-party models you do not retrain
- C. It uses Python
- D. There is no data
---
Answer Key and Explanations
Check these only after attempting every question.
1. B - The same input can produce different outputs, so pass/fail assertions alone are insufficient
Reliability becomes distributional - you measure rates and regressions, not just single-run correctness.
2. B - A handful of favourable examples says nothing about failure rate on real traffic
Without a measured denominator you have anecdotes, not reliability.
3. B - Per use case, based on the cost of a wrong answer
A brainstorming assistant and a medical summariser warrant very different thresholds.
4. B - Confidently wrong output that no one flags
Infrastructure failures are loud; silent quality failures are the ones that reach users.
5. B - Behaviour is driven by prompts, context, and third-party models you do not retrain
Your levers are prompts, retrieval, and guardrails - with the model as a changing dependency.
Practical Check
Apply this module to your own work: complete the module activity for *What Reliability Means for LLM Applications*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.