Module 11 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. Components should be tested separately because…
- A. It is tidier
- B. A failure could originate in loading, splitting, retrieval, or generation, and the fixes differ
- C. Tests run faster
- D. It is required
2. A deterministic unit test is possible for…
- A. Model creativity
- B. Splitters, parsers, schema validation, and retrieval filters
- C. Answer quality
- D. Tone
3. Using an LLM to grade LLM output requires care because…
- A. It is slow
- B. A judge sharing the generator's blind spots inflates scores on exactly the wrong cases
- C. It costs more
- D. It needs a schema
4. A useful eval set includes…
- A. Only typical questions
- B. Typical, edge, unanswerable, and adversarial cases
- C. Only failures
- D. Only long inputs
5. Evals should be re-run when…
- A. Only before launch
- B. The prompt, model, retrieval settings, or corpus changes
- C. Monthly regardless
- D. Never
---
Answer Key and Explanations
Check these only after attempting every question.
1. B - A failure could originate in loading, splitting, retrieval, or generation, and the fixes differ
End-to-end-only testing tells you something broke, not which stage to repair.
2. B - Splitters, parsers, schema validation, and retrieval filters
Test the deterministic parts deterministically; evaluate the probabilistic parts statistically.
3. B - A judge sharing the generator's blind spots inflates scores on exactly the wrong cases
Use a different model, human spot-checks, or objective criteria wherever feasible.
4. B - Typical, edge, unanswerable, and adversarial cases
The dangerous behaviours only appear when you deliberately test for them.
5. B - The prompt, model, retrieval settings, or corpus changes
Each of those silently changes behaviour; the eval set is your regression test.
Practical Check
Apply this module to your own work: complete the module activity for *Testing and Evaluating LangChain Components*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.