Practice workbook · 36 lesson exercises

Attempt each lesson before reading the separate solution key.

01 · Read a tensor before training a model

Complete these before opening the solutions.

01A · A new observation: use X = [[2, 0, -1]] with the worked weights and biases. Predict the shape and calculate both outputs by hand; then verify them with PyTorch.
01B · A broadcasting trap: create predictions [[1.0], [3.0], [5.0]] and targets [2.0, 4.0, 6.0]. Inspect the subtraction's shape. Repair the target shape and explain which comparisons now occur.
01C · Count the adjustable numbers: a layer maps four features to three outputs with a bias. State the input, weight, bias and output shapes for a batch of five, and count its learned scalar parameters.

02A · Follow the inactive path: repeat the scalar forward pass with x = -2, leaving all parameters and the target unchanged. Calculate z, h, yhat, half-squared loss and default MSE.
02B · Write three contracts: specify output shape, target shape, target dtype and loss for a batch of six house-price predictions, six binary ring labels, and six three-class observations.
02C · Confidence changes loss: using logits [log(2), 0, 0], calculate the loss when the target is class two. Explain why the prediction is unchanged but the loss differs from the worked class-zero case.

03A · A different target: change only the target to y = -0.5. Derive the original-position loss and all four parameter gradients, then check them with finite differences.
03B · A closed gate: set w = -0.5, b = 0.1, keeping x = 2 and the original remaining values. Determine which gradients are zero and explain each zero through its path.
03C · Compare step sizes: from the original parameters, calculate one simultaneous update at learning rates 0.01, 0.1 and 1.0. Evaluate each new loss before interpreting which step helped most.

04A · Audit membership: write checks that the three split-ID sets are pairwise disjoint, have sizes 60/20/20, and together contain exactly IDs 0 through 99. Deliberately insert a duplicate across splits and confirm a check fails.
04B · Catch a leaking scaler: training values are [2, 4, 6] and one validation value is 20. Calculate train-only mean and population standard deviation, then transform 20. Compare with a scaler fitted to all four values and explain the information leak.
04C · Count batches: for 35 training observations and batch size eight, predict the batch sizes with drop_last=False and True. State which setting you would use for complete evaluation and why.

05A · Explain the broken order: consider loss.backward(); optimizer.zero_grad(set_to_none=True); optimizer.step(). Explain what information is discarded and write a correct order for one ordinary update.
05B · Weight the final batch: three batches contain four, four and two observations, with mean losses 0.5, 1.0 and 3.0. Compute the full-dataset mean and compare it with the unweighted mean of batch means.
05C · Verify a mode distinction: use a Linear(1, 1) model in evaluation mode on an input while gradient recording is enabled. Inspect the output's requires_grad. Repeat inside torch.no_grad() and explain the difference.

06A · Count a smaller candidate: count the parameters in Linear(2, 1) and in a 2–16–1 MLP with ReLU. Compare the latter with the displayed 2–16–16–1 model. Explain which additional capability depends on ReLU.
06B · Explain the boundary: sketch ideal circles of radii one and two. Draw a straight decision boundary, identify where it must fail, and describe why a collection of piecewise-linear regions could help.
06C · Make a selection plan: specify a comparison of the linear and MLP ring classifiers, including fixed data, training-only preprocessing, one primary validation metric, a stopping rule and the point at which the test set may be evaluated. Do not invent performance numbers.

7A · Dropout expectation. With dropout probability 0.4 and input value 6, list the possible training outputs and their probabilities. Compute their expected value and the evaluation output.
7B · Choose a checkpoint. Validation losses at epochs 10, 20, 30, 40 are 0.48, 0.31, 0.35, 0.39. Training losses keep falling. Which checkpoint does the declared minimum-loss rule choose? Why is “epoch 40 has lowest training loss” insufficient?
7C · A controlled experiment. Read the noisy-case configuration and final history rows. Write a four-sentence comparison naming sample size, candidate differences, best validation epoch and the limitation created by changing several regularization choices together.

8A · Broadcast trap. Write the 3-by-3 difference matrix for predictions and targets 1, 2, 3. Calculate its MSE and explain why a diagonal-only comparison is intended.
8B · Update or no update? Name three distinct causes of unchanged parameters and a concrete observation that distinguishes each.
8C · Confusion counts. For TN=60, FP=10, FN=20, TP=30, calculate accuracy, precision and recall. What changes if the cost of false negatives rises?

9A · Local detector. Compute the horizontal and vertical kernel responses on a vertical center-column patch. Assume zero bias.
9B · Shape change. Remove convolution padding but keep the rest of the architecture. Find the new feature shape after pooling and the required input dimension of the final linear layer.
9C · Label-preserving transforms. Classify horizontal flip, 90-degree rotation and mild added sensor noise as safe, label-changing or assumption-dependent for this dataset. Explain each decision.

10A · Missing record. A teammate saves only a model state dictionary and a test score. Name five missing pieces required to understand or reproduce the experiment.
10B · Best weights. Explain why storing best_state = model.state_dict() without a deep copy can be dangerous during continued training. Identify the relevant line in our implementation.
10C · Same test twice. You rerun the frozen reference experiment and obtain the same accuracy. Does the effective test sample become 240? Explain what this rerun actually verifies.

11A · Freeze is not eval. A frozen feature extractor contains dropout. Why can repeated outputs still differ? What two separate settings should you inspect?
11B · Equal attention. If both scores are zero and values remain [2,0] and [0,4], compute weights and output. Explain why the output is not either original value.
11C · Future leakage. A model predicting the next token allows the current position to attend to later ground-truth tokens during training. Explain why this changes the prediction problem and how a causal mask addresses the information path.

12A · Protocol audit. A learner compares four candidates on test accuracy and reports only the best. Rewrite the procedure to preserve a final assessment boundary.
12B · Evidence package. List the files a reviewer needs to reproduce your training/validation selection and understand the final result. Distinguish them from a virtual environment or downloaded wheel, which should not be submitted.
12C · Responsible conclusion. Write a two-sentence conclusion that includes task, split, baseline comparison and one limitation. Use your actual measured results, not a promised improvement.