Module 07 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. The minimum useful trace records…
- A. The final answer
- B. Resolved prompt, retrieved context, tool calls, and output for each step
- C. Latency only
- D. The model name
2. Aggregate metrics without traces leave you able to…
- A. Fix issues quickly
- B. See that quality dropped, but not why
- C. Prevent incidents
- D. Grade answers
3. Logging user prompts and retrieved documents requires…
- A. Nothing extra
- B. Retention limits, access control, and often redaction
- C. Only encryption
- D. More storage only
4. Sampling traces in high-volume production is reasonable provided…
- A. It is random only
- B. Errors and flagged cases are always captured, not just a random slice
- C. You sample 1%
- D. You disable metrics
5. User feedback signals should be…
- A. Ignored
- B. Captured and joined to traces so complaints can be investigated concretely
- C. Counted only
- D. Shown publicly
---
Answer Key and Explanations
Check these only after attempting every question.
1. B - Resolved prompt, retrieved context, tool calls, and output for each step
Debugging requires seeing what was actually sent, not what you believe was sent.
2. B - See that quality dropped, but not why
Metrics detect; traces diagnose. You need both.
3. B - Retention limits, access control, and often redaction
These logs become a searchable copy of user content and corpus material.
4. B - Errors and flagged cases are always captured, not just a random slice
Uniform sampling systematically misses rare, important failures.
5. B - Captured and joined to traces so complaints can be investigated concretely
A thumbs-down with no linked trace is a statistic; with a trace it is a fixable bug.
Practical Check
Apply this module to your own work: complete the module activity for *Observability: Traces, Metrics, and Logs*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.