Deep Learning · Module 11
Answers and explanations · Two bridges beyond the core: reuse and attention
Compare these references after your own attempt. You can read them at any time; they do not record a live model or vendor run.
Deep Learning · Module 11
Compare these references after your own attempt. You can read them at any time; they do not record a live model or vendor run.
11A. Frozen weights stop parameter gradients, while active dropout still randomly changes intermediate values. Inspect requires_grad/optimizer membership separately from module train/eval mode. Keep a fixed extractor in eval while a trainable head is in train mode.
11B. Equal scores give weights [0.5,0.5]. Weighted output is [1,2]. Attention forms a mixture; it need not select one whole value vector.
11C. Future ground-truth tokens reveal information unavailable at causal prediction time, making training/evaluation artificially easier. A causal mask prevents attention to those future positions before normalization. Correct masking is necessary but not alone sufficient for a complete language model.