Practice and capstone

Practice workbook · 36 lesson exercises

Attempt each lesson before reading the separate solution key.

01 · Read a tensor before training a model

Complete these before opening the solutions.

• 01A · A new observation: use X = [[2, 0, -1]] with the worked weights and biases. Predict the shape and calculate both outputs by hand; then verify them with PyTorch.
• 01B · A broadcasting trap: create predictions [[1.0], [3.0], [5.0]] and targets [2.0, 4.0, 6.0]. Inspect the subtraction's shape. Repair the target shape and explain which comparisons now occur.
• 01C · Count the adjustable numbers: a layer maps four features to three outputs with a bias. State the input, weight, bias and output shapes for a batch of five, and count its learned scalar parameters.

02 · Follow a prediction into its loss

• 02A · Follow the inactive path: repeat the scalar forward pass with x = -2, leaving all parameters and the target unchanged. Calculate z, h, yhat, half-squared loss and default MSE.
• 02B · Write three contracts: specify output shape, target shape, target dtype and loss for a batch of six house-price predictions, six binary ring labels, and six three-class observations.
• 02C · Confidence changes loss: using logits [log(2), 0, 0], calculate the loss when the target is class two. Explain why the prediction is unchanged but the loss differs from the worked class-zero case.

03 · Trace responsibility backward

• 03A · A different target: change only the target to y = -0.5. Derive the original-position loss and all four parameter gradients, then check them with finite differences.
• 03B · A closed gate: set w = -0.5, b = 0.1, keeping x = 2 and the original remaining values. Determine which gradients are zero and explain each zero through its path.
• 03C · Compare step sizes: from the original parameters, calculate one simultaneous update at learning rates 0.01, 0.1 and 1.0. Evaluate each new loss before interpreting which step helped most.

04 · Decide what the model is allowed to learn from

• 04A · Audit membership: write checks that the three split-ID sets are pairwise disjoint, have sizes 60/20/20, and together contain exactly IDs 0 through 99. Deliberately insert a duplicate across splits and confirm a check fails.
• 04B · Catch a leaking scaler: training values are [2, 4, 6] and one validation value is 20. Calculate train-only mean and population standard deviation, then transform 20. Compare with a scaler fitted to all four values and explain the information leak.
• 04C · Count batches: for 35 training observations and batch size eight, predict the batch sizes with drop_last=False and True. State which setting you would use for complete evaluation and why.

05 · Give each training-loop line a job

• 05A · Explain the broken order: consider loss.backward(); optimizer.zero_grad(set_to_none=True); optimizer.step(). Explain what information is discarded and write a correct order for one ordinary update.
• 05B · Weight the final batch: three batches contain four, four and two observations, with mean losses 0.5, 1.0 and 3.0. Compute the full-dataset mean and compare it with the unweighted mean of batch means.
• 05C · Verify a mode distinction: use a Linear(1, 1) model in evaluation mode on an input while gradient recording is enabled. Inspect the output's requires_grad. Repeat inside torch.no_grad() and explain the difference.

06 · Ask what extra flexibility actually buys

• 06A · Count a smaller candidate: count the parameters in Linear(2, 1) and in a 2–16–1 MLP with ReLU. Compare the latter with the displayed 2–16–16–1 model. Explain which additional capability depends on ReLU.
• 06B · Explain the boundary: sketch ideal circles of radii one and two. Draw a straight decision boundary, identify where it must fail, and describe why a collection of piecewise-linear regions could help.
• 06C · Make a selection plan: specify a comparison of the linear and MLP ring classifiers, including fixed data, training-only preprocessing, one primary validation metric, a stopping rule and the point at which the test set may be evaluated. Do not invent performance numbers.

07 · When fitting better means predicting worse

• 7A · Dropout expectation. With dropout probability 0.4 and input value 6, list the possible training outputs and their probabilities. Compute their expected value and the evaluation output.
• 7B · Choose a checkpoint. Validation losses at epochs 10, 20, 30, 40 are 0.48, 0.31, 0.35, 0.39. Training losses keep falling. Which checkpoint does the declared minimum-loss rule choose? Why is “epoch 40 has lowest training loss” insufficient?
• 7C · A controlled experiment. Read the noisy-case configuration and final history rows. Write a four-sentence comparison naming sample size, candidate differences, best validation epoch and the limitation created by changing several regularization choices together.

08 · Debug the learning process before enlarging the model

• 8A · Broadcast trap. Write the 3-by-3 difference matrix for predictions and targets 1, 2, 3. Calculate its MSE and explain why a diagonal-only comparison is intended.
• 8B · Update or no update? Name three distinct causes of unchanged parameters and a concrete observation that distinguishes each.
• 8C · Confusion counts. For TN=60, FP=10, FN=20, TP=30, calculate accuracy, precision and recall. What changes if the cost of false negatives rises?

09 · A tiny convolutional network you can account for

• 9A · Local detector. Compute the horizontal and vertical kernel responses on a vertical center-column patch. Assume zero bias.
• 9B · Shape change. Remove convolution padding but keep the rest of the architecture. Find the new feature shape after pooling and the required input dimension of the final linear layer.
• 9C · Label-preserving transforms. Classify horizontal flip, 90-degree rotation and mild added sensor noise as safe, label-changing or assumption-dependent for this dataset. Explain each decision.

10 · Turn a successful run into a reproducible experiment

• 10A · Missing record. A teammate saves only a model state dictionary and a test score. Name five missing pieces required to understand or reproduce the experiment.
• 10B · Best weights. Explain why storing best_state = model.state_dict() without a deep copy can be dangerous during continued training. Identify the relevant line in our implementation.
• 10C · Same test twice. You rerun the frozen reference experiment and obtain the same accuracy. Does the effective test sample become 240? Explain what this rerun actually verifies.

11 · Two bridges beyond the core: reuse and attention

• 11A · Freeze is not eval. A frozen feature extractor contains dropout. Why can repeated outputs still differ? What two separate settings should you inspect?
• 11B · Equal attention. If both scores are zero and values remain [2,0] and [0,4], compute weights and output. Explain why the output is not either original value.
• 11C · Future leakage. A model predicting the next token allows the current position to attend to later ground-truth tokens during training. Explain why this changes the prediction problem and how a causal mask addresses the information path.

12 · Defend one small neural experiment end to end

• 12A · Protocol audit. A learner compares four candidates on test accuracy and reports only the best. Rewrite the procedure to preserve a final assessment boundary.
• 12B · Evidence package. List the files a reviewer needs to reproduce your training/validation selection and understand the final result. Distinguish them from a virtual environment or downloaded wheel, which should not be submitted.
• 12C · Responsible conclusion. Write a two-sentence conclusion that includes task, split, baseline comparison and one limitation. Use your actual measured results, not a promised improvement.

Assessments · attempt before opening the answer key

Readiness diagnostic

R1. For predictions 2 and 5, with targets 1 and 3, compute MSE. R2. Why must a fitted scaler use training rows only? R3. A dataset contains repeated readings from each machine. Why can a random row split mislead? R4. Write the output shape for a batch of seven rows and a linear layer mapping three features to two outputs. R5. Is choosing the best test score among ten models a valid final test? Explain.

Readiness is about finding review needs. If two or more answers are missing, revisit Python arrays, Statistics residuals and Applied ML evaluation before starting the training modules.

Checkpoint A · computation (after Lesson 04)

A1. Distinguish a batch axis from a feature axis using [16,3]. A2. Predict the result shape of [4,1] - [4] and explain the likely error. A3. Why can two affine layers without an activation be replaced by one affine layer? A4. In the hand-worked network, give dL/dw and explain its sign. A5. Why use float64 and a small but nonzero finite-difference step for a gradient check? A6. With 360 training rows and batch size 32, how many updates occur in one complete epoch without dropping rows?

Checkpoint B · learning (after Lesson 08)

B1. Put zero_grad, forward/loss, backward and step in order and explain each. B2. What does eval change, and what does no_grad change? B3. A model's training loss falls but validation loss rises. Give two plausible explanations, one of which is not overfitting. B4. How is a best-checkpoint model selected in this package, and does that save training computation? B5. For L=0.5*w², why does SGD learning rate 2.2 produce growing loss? B6. If a classifier predicts no positives, how does this report define precision? Why should that convention be disclosed?

Checkpoint C · extension and evidence (after Lesson 12)

C1. Derive the 170-parameter CNN count. C2. Why does a 90-degree rotation require changing the bars label? C3. Does freezing weights disable dropout? Explain. C4. Compute equal-score attention for values [2,0] and [0,4]. C5. Does repeating the same test run increase its sample size? C6. A synthetic test has zero errors out of 120. Name two conclusions this does not establish.

Suggested self-assessment: 2 points per complete explanation, 1 for a correct conclusion with incomplete reasoning, 0 for an incorrect or unsupported conclusion. A checkpoint score is feedback, not automatic certification. Rework any loss-shape, leakage or test-selection mistake before continuing.

Capstone · one defensible experiment

Deliverables

1. protocol.md: one-row meaning, available features, target, IID assumption, alternative group/time split if needed, fixed candidate budget, seeds, primary selection metric and test boundary
2. Completed exercises/stages.py, exercises/capstone.py and public-check output
3. Validation-only selection output and one learning-curve visualization with text interpretation
4. Final assessment output after selection is frozen; report actual counts, metric units and naive baseline
5. A 600–900-word report explaining shape flow, one gradient calculation, selection decision, validation error analysis, reproducibility and limits

Do not submit virtual environments, downloaded wheels, caches or secret credentials. The source, small data/outputs and report are sufficient. The bundled data and reference test outcomes are visible; disclose prior exposure. For genuine learner assessment, an instructor should generate a new held-out seed after the model is frozen.

Commands and stopping boundary

python check_learner.py
python -m exercises.capstone
# Read outputs/learner_capstone/selection.json and freeze your written plan.
python -m exercises.capstone --final

--final requires an existing selection record whose configuration/rationale hash matches the current plan. It retrains the selected model reproducibly, checks the validation result, then evaluates test once and creates final.json exclusively. A second final invocation refuses to overwrite the assessment. Deleting files to keep tuning on the same test is not a valid fresh assessment.

The example reference path is separate:

python -m solutions.capstone
python -m solutions.capstone --final

The source package includes the author's reference results. Those two reference commands will therefore refuse to overwrite the existing final report until you choose a separate output folder. To reproduce unchanged reference mechanics, use --output-dir outputs/my_reference_reproduction on both commands. That rerun is a reproducibility check, not additional test evidence.

Rubric: 100 points

• Question and data contract, 15: explicit row/target, available inputs, IID assumption and deployment limits
• Evaluation integrity, 20: disjoint IDs, train-only fit, validation-only choices, no test-driven revisions
• Implementation, 20: shapes/dtypes/losses correct, gradients cleared, weights update, eval/no_grad correct
• Baselines and selection, 15: train-mean and linear baseline, bounded candidate set, best-checkpoint evidence
• Diagnostics, 10: one validation error pattern, a plausible alternative explanation and discriminating next check
• Reproducibility, 10: versions/seeds/configs, source runnable, checks reported accurately
• Communication, 10: actual measured values/counts, uncertainty and limits, no unsupported mastery or deployment claims

Suggested pass: at least 70 points and all integrity gates. Gate failures require correction regardless of score: split overlap, preprocessing fitted on held-out rows, test-tuned model choice, invented results or a mismatch between code and claimed execution. A high accuracy alone earns no exemption.

Optional CNN extension: compare the supplied flattened linear model and 170-parameter CNN using validation only. Explain every shape and which augmentations preserve the target. Transfer learning and attention are conceptual extensions, not required capstone deliverables.