# Practice workbook · 36 lesson exercises

Attempt each lesson before reading the separate solution key.

## 01 · Read a tensor before training a model

Complete these before opening the solutions.

- **01A · A new observation:** use `X = [[2, 0, -1]]` with the worked weights and biases. Predict the shape and calculate both outputs by hand; then verify them with PyTorch.
- **01B · A broadcasting trap:** create predictions `[[1.0], [3.0], [5.0]]` and targets `[2.0, 4.0, 6.0]`. Inspect the subtraction's shape. Repair the target shape and explain which comparisons now occur.
- **01C · Count the adjustable numbers:** a layer maps four features to three outputs with a bias. State the input, weight, bias and output shapes for a batch of five, and count its learned scalar parameters.

## 02 · Follow a prediction into its loss

- **02A · Follow the inactive path:** repeat the scalar forward pass with `x = -2`, leaving all parameters and the target unchanged. Calculate `z`, `h`, `yhat`, half-squared loss and default MSE.
- **02B · Write three contracts:** specify output shape, target shape, target dtype and loss for a batch of six house-price predictions, six binary ring labels, and six three-class observations.
- **02C · Confidence changes loss:** using logits `[log(2), 0, 0]`, calculate the loss when the target is class two. Explain why the prediction is unchanged but the loss differs from the worked class-zero case.

## 03 · Trace responsibility backward

- **03A · A different target:** change only the target to `y = -0.5`. Derive the original-position loss and all four parameter gradients, then check them with finite differences.
- **03B · A closed gate:** set `w = -0.5`, `b = 0.1`, keeping `x = 2` and the original remaining values. Determine which gradients are zero and explain each zero through its path.
- **03C · Compare step sizes:** from the original parameters, calculate one simultaneous update at learning rates `0.01`, `0.1` and `1.0`. Evaluate each new loss before interpreting which step helped most.

## 04 · Decide what the model is allowed to learn from

- **04A · Audit membership:** write checks that the three split-ID sets are pairwise disjoint, have sizes 60/20/20, and together contain exactly IDs 0 through 99. Deliberately insert a duplicate across splits and confirm a check fails.
- **04B · Catch a leaking scaler:** training values are `[2, 4, 6]` and one validation value is `20`. Calculate train-only mean and population standard deviation, then transform 20. Compare with a scaler fitted to all four values and explain the information leak.
- **04C · Count batches:** for 35 training observations and batch size eight, predict the batch sizes with `drop_last=False` and `True`. State which setting you would use for complete evaluation and why.

## 05 · Give each training-loop line a job

- **05A · Explain the broken order:** consider `loss.backward(); optimizer.zero_grad(set_to_none=True); optimizer.step()`. Explain what information is discarded and write a correct order for one ordinary update.
- **05B · Weight the final batch:** three batches contain four, four and two observations, with mean losses `0.5`, `1.0` and `3.0`. Compute the full-dataset mean and compare it with the unweighted mean of batch means.
- **05C · Verify a mode distinction:** use a `Linear(1, 1)` model in evaluation mode on an input while gradient recording is enabled. Inspect the output's `requires_grad`. Repeat inside `torch.no_grad()` and explain the difference.

## 06 · Ask what extra flexibility actually buys

- **06A · Count a smaller candidate:** count the parameters in `Linear(2, 1)` and in a 2–16–1 MLP with ReLU. Compare the latter with the displayed 2–16–16–1 model. Explain which additional capability depends on ReLU.
- **06B · Explain the boundary:** sketch ideal circles of radii one and two. Draw a straight decision boundary, identify where it must fail, and describe why a collection of piecewise-linear regions could help.
- **06C · Make a selection plan:** specify a comparison of the linear and MLP ring classifiers, including fixed data, training-only preprocessing, one primary validation metric, a stopping rule and the point at which the test set may be evaluated. Do not invent performance numbers.

## 07 · When fitting better means predicting worse

- **7A · Dropout expectation.** With dropout probability 0.4 and input value 6, list the possible training outputs and their probabilities. Compute their expected value and the evaluation output.
- **7B · Choose a checkpoint.** Validation losses at epochs 10, 20, 30, 40 are 0.48, 0.31, 0.35, 0.39. Training losses keep falling. Which checkpoint does the declared minimum-loss rule choose? Why is “epoch 40 has lowest training loss” insufficient?
- **7C · A controlled experiment.** Read the noisy-case configuration and final history rows. Write a four-sentence comparison naming sample size, candidate differences, best validation epoch and the limitation created by changing several regularization choices together.

## 08 · Debug the learning process before enlarging the model

- **8A · Broadcast trap.** Write the 3-by-3 difference matrix for predictions and targets 1, 2, 3. Calculate its MSE and explain why a diagonal-only comparison is intended.
- **8B · Update or no update?** Name three distinct causes of unchanged parameters and a concrete observation that distinguishes each.
- **8C · Confusion counts.** For TN=60, FP=10, FN=20, TP=30, calculate accuracy, precision and recall. What changes if the cost of false negatives rises?

## 09 · A tiny convolutional network you can account for

- **9A · Local detector.** Compute the horizontal and vertical kernel responses on a vertical center-column patch. Assume zero bias.
- **9B · Shape change.** Remove convolution padding but keep the rest of the architecture. Find the new feature shape after pooling and the required input dimension of the final linear layer.
- **9C · Label-preserving transforms.** Classify horizontal flip, 90-degree rotation and mild added sensor noise as safe, label-changing or assumption-dependent for this dataset. Explain each decision.

## 10 · Turn a successful run into a reproducible experiment

- **10A · Missing record.** A teammate saves only a model state dictionary and a test score. Name five missing pieces required to understand or reproduce the experiment.
- **10B · Best weights.** Explain why storing `best_state = model.state_dict()` without a deep copy can be dangerous during continued training. Identify the relevant line in our implementation.
- **10C · Same test twice.** You rerun the frozen reference experiment and obtain the same accuracy. Does the effective test sample become 240? Explain what this rerun actually verifies.

## 11 · Two bridges beyond the core: reuse and attention

- **11A · Freeze is not eval.** A frozen feature extractor contains dropout. Why can repeated outputs still differ? What two separate settings should you inspect?
- **11B · Equal attention.** If both scores are zero and values remain `[2,0]` and `[0,4]`, compute weights and output. Explain why the output is not either original value.
- **11C · Future leakage.** A model predicting the next token allows the current position to attend to later ground-truth tokens during training. Explain why this changes the prediction problem and how a causal mask addresses the information path.

## 12 · Defend one small neural experiment end to end

- **12A · Protocol audit.** A learner compares four candidates on test accuracy and reports only the best. Rewrite the procedure to preserve a final assessment boundary.
- **12B · Evidence package.** List the files a reviewer needs to reproduce your training/validation selection and understand the final result. Distinguish them from a virtual environment or downloaded wheel, which should not be submitted.
- **12C · Responsible conclusion.** Write a two-sentence conclusion that includes task, split, baseline comparison and one limitation. Use your actual measured results, not a promised improvement.