Module 04 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. Which part of an LLM app can be tested fully deterministically?
- A. Answer quality
- B. Schema validation, tool argument handling, and retrieval filters
- C. Tone
- D. Creativity
2. A prompt change should be accompanied by…
- A. A deploy
- B. Re-running the eval set to detect regressions elsewhere
- C. A comment
- D. A version bump only
3. Testing tool functions independently of the model matters because…
- A. It is faster
- B. It separates 'the tool is broken' from 'the model called it wrongly'
- C. Tools are simple
- D. Models are slow
4. A test that asserts an exact model output string will…
- A. Be stable
- B. Be flaky and get disabled, removing coverage
- C. Improve quality
- D. Run faster
5. Mocking the model in unit tests is appropriate when…
- A. Always
- B. You are testing the surrounding logic - parsing, routing, error handling - not the model
- C. Never
- D. Only in production
---
Answer Key and Explanations
Check these only after attempting every question.
1. B - Schema validation, tool argument handling, and retrieval filters
Deterministic surfaces deserve ordinary unit tests; save statistical evaluation for the model's behaviour.
2. B - Re-running the eval set to detect regressions elsewhere
Prompts are code. An untested prompt edit is an untested deploy.
3. B - It separates 'the tool is broken' from 'the model called it wrongly'
Two very different fixes; you need to know which one you are looking at.
4. B - Be flaky and get disabled, removing coverage
Brittle assertions erode trust in the suite, which is worse than having no assertion.
5. B - You are testing the surrounding logic - parsing, routing, error handling - not the model
Mock to isolate your code; use real calls in evaluation to measure behaviour.
Practical Check
Apply this module to your own work: complete the module activity for *Unit Tests for Prompts, Schemas, and Tools*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.