Answers and explanations · Defend one small neural experiment end to end

Lesson 12

12A. Compare candidates on training/validation only, fix the selected model and decision rules, then evaluate once on untouched test. If test has already driven choice, disclose contamination and obtain a new appropriate held-out assessment; do not relabel the old score as final evidence.

12B. Submit source, protocol, dependency pins, data generator/split IDs, completed stages, selection/final JSON, curve and report. Exclude environments/caches/wheels. This retains the reproducible decisions without distributing hundreds of megabytes of runtime binaries.

12C. Example using this package's recorded regression run: “On 120 held-out synthetic regression samples, the validation-selected MLP achieved RMSE 0.1612 target units, compared with 2.2854 for the train-mean baseline. This is one fixed generated experiment, with visible reference outcomes and no real deployment population, so it does not establish real sensor performance.” Verify the exact baseline against your own run; change the sentence when your results differ.

Readiness answers

R1: ((2-1)²+(5-3)²)/2=2.5. R2: held-out information must not influence fitted preprocessing. R3: repeated machine-specific patterns can leak across a row split; split by machine or time according to deployment. R4: [7,2]. R5: no; test has become a selection set.

Checkpoint answers

A1: 16 observations, 3 features per observation. A2: [4,4], likely unintended all-pairs comparisons. A3: substitution composes matrices and biases into one affine map. A4: 2.8; locally increasing w increases loss at the original point, so gradient descent decreases w. A5: float64 reduces rounding error; too-small differences cancel, while too-large steps are poor local approximations; avoid ReLU kinks. A6: 12 updates, eleven batches of 32 and one of 8.

B1: clear accumulated gradients; compute prediction/loss; backpropagate derivatives; update parameters. B2: eval changes mode-sensitive layers, no_grad suppresses graph recording. B3: overfit or a distribution/label-quality difference between partitions; also check comparable measurement modes. B4: minimum validation-loss checkpoint within the full epoch budget; computation does not stop early. B5: magnitude multiplies by 1.2, squared loss by 1.44. B6: zero by convention; undefined-denominator conventions affect reported summaries and should be explicit.

C1: convolution 4×(1×3×3+1)=40; head 64×2+2=130; total170. C2: orientation is the target. C3: no; module mode is separate. C4: weights [0.5,0.5], result [1,2]. C5: no, same observations. C6: it does not prove error-free future performance, real-image generalization, fairness, safety or calibration. Any two well-explained unsupported claims suffice.

Staged code explanations

solutions/stages.py contains the five implementations. Stage 1 uses matrix multiplication plus bias. Stage 2 fits mean and population standard deviation only on supplied training rows, clamping constant-column scales. Stage 3 returns raw logits with ReLU hidden features. Stage 4 clears, computes, backpropagates and steps in the right order. Stage 5 changes module mode and disables gradient recording before converting logits to probabilities. The checker defaults to your starter and must fail until you implement these operations.