Module 11 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. The train/validation gap narrows as you add data and validation keeps improving. What does that suggest?
- A. The model is the constraint
- B. More data will keep helping
- C. The model is overfitting badly
- D. The learning rate is wrong
2. Predictions in the 0.8-1.0 confidence bucket are correct 60% of the time. What is wrong?
- A. Accuracy is too low
- B. The model is not calibrated, so its probabilities cannot drive a threshold
- C. The bucket is too wide
- D. Nothing
3. A saliency map highlights feature 3. What can you conclude?
- A. Feature 3 causes the outcome
- B. The model is locally sensitive to feature 3 at this input
- C. Feature 3 is the most important feature overall
- D. Feature 3 should be removed
4. Under distribution shift, what happens to accuracy and confidence?
- A. Both fall
- B. Accuracy falls while confidence stays high
- C. Both rise
- D. Confidence falls first
5. What does a model card most need that is usually missing?
- A. A parameter count
- B. An out-of-scope section naming where it must not be used
- C. Training time
- D. A licence
---
Answer Key and Explanations
Check these only after attempting every question.
1. B — More data will keep helping
A stable gap with flat validation would mean the opposite: the model, not the data, is the limit.
2. B — The model is not calibrated, so its probabilities cannot drive a threshold
'Act above 0.8' means something quite different on a miscalibrated model. Check before building a decision rule on the number.
3. B — The model is locally sensitive to feature 3 at this input
It is a first-order approximation at one point for one example. Treat it as a lead, then confirm by intervening.
4. B — Accuracy falls while confidence stays high
That divergence is what makes shift dangerous, and why you cannot use the model's own probability to decide whether to trust it.
5. B — An out-of-scope section naming where it must not be used
Without it the card documents nothing that would stop a misuse, which is the main thing it exists to do.
Practical Check
Apply this module to your own work: complete the module activity for *Diagnostics, Interpretability, Robustness, and Responsible Use*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.