Module 09 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. What do the residual connections in a transformer block make possible?
- A. Smaller models
- B. Gradients reaching early layers even when intermediate transformations are unhelpful
- C. Faster tokenisation
- D. Larger batch sizes
2. A transformer block's output shape compared to its input shape is:
- A. Half the size
- B. The same
- C. Double
- D. Depends on the number of heads
3. What is the cheapest useful way to apply a pretrained encoder?
- A. Fine-tune end to end
- B. Freeze it, embed once, cache the vectors, and fit a small head
- C. Train it from scratch
- D. Distil it into a smaller model
4. When should you decide which slices to report a metric over?
- A. After seeing the results
- B. Before looking at the results
- C. Only if the overall metric is poor
- D. Slicing is unnecessary
5. Your report says accuracy 0.91 on random vectors rather than real text. What must the write-up state?
- A. Nothing extra
- B. That this measures the pipeline, not language quality
- C. That the model is production-ready
- D. The GPU used
---
Answer Key and Explanations
Check these only after attempting every question.
1. B — Gradients reaching early layers even when intermediate transformations are unhelpful
The input has a path straight to the output through the addition, which is what makes stacking many blocks trainable.
2. B — The same
A block does not resize the representation; it enriches each token with context from the others.
3. B — Freeze it, embed once, cache the vectors, and fit a small head
The expensive step happens once, and downstream experiments become cheap enough to iterate on.
4. B — Before looking at the results
Choosing slices afterwards means selecting the split that tells the story you want, which is not a measurement.
5. B — That this measures the pipeline, not language quality
The numbers are real; what they measure is that the code runs end to end, which is a much smaller claim.
Practical Check
Apply this module to your own work: complete the module activity for *Transformers for Text Classification and Feature Extraction*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.