Module 05 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. A model 'is not learning'. What should you change first?
- A. The architecture
- B. The learning rate
- C. The loss function
- D. The batch size
2. What does momentum add to plain SGD?
- A. A per-parameter step size
- B. A running average of past gradients that carries updates through flat regions
- C. Automatic learning rate decay
- D. Gradient clipping
3. In a 12-layer network with poorly scaled initialisation, what do you observe?
- A. Identical gradients at every layer
- B. The first layer's gradient is orders of magnitude off the last layer's
- C. Training is simply slower
- D. The loss is always NaN
4. Why does BatchNorm behave differently in train and eval mode?
- A. It is disabled at inference
- B. It uses batch statistics while training and stored running statistics at inference
- C. It only normalises during backpropagation
- D. It switches to LayerNorm
5. You add dropout and training loss rises. What does that mean?
- A. Dropout is misconfigured
- B. The regularisation is working as intended
- C. The learning rate is too low
- D. The model is underfitting
---
Answer Key and Explanations
Check these only after attempting every question.
1. B — The learning rate
It fails in two opposite directions — crawling and diverging — and both are visible immediately. People rebuild models to fix what one hyperparameter caused.
2. B — A running average of past gradients that carries updates through flat regions
Adam is the one that adapts a per-parameter step size from gradient history.
3. B — The first layer's gradient is orders of magnitude off the last layer's
Each layer's scaling compounds multiplicatively through depth, so early layers receive almost nothing or explode.
4. B — It uses batch statistics while training and stored running statistics at inference
Forgetting model.eval() makes the score depend on how the validation set happened to be batched.
5. B — The regularisation is working as intended
Regularisation should raise training loss. If it does not, it is not doing anything.
Practical Check
Apply this module to your own work: complete the module activity for *Optimization, Initialization, Normalization, and Regularization*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.