Module 03 Activity

Scenario

A training loop that runs is not a training loop that works. Build one whose correctness you can prove.

Compute a forward pass and a loss. Confirm the loss is a scalar with a grad_fn.
Verify a gradient by hand: for y = w * x with x = 3, w.grad must be 3.
Work the chain-rule example on paper, then check a.grad and b.grad against your arithmetic.
Demonstrate gradient accumulation by calling .backward() twice without clearing.
Train on data generated by a known rule and confirm the learned parameters recover it.
Checkpoint on validation loss and reload, confirming the restored score matches.

A training script plus a note recording the hand-calculated gradients and the recovered coefficients.

a.grad is 150.0 and b.grad is 90.0 for the worked example.
Two backward passes without clearing give exactly double the gradient.
Training on y = 2x0 - 3x1 + 1 recovers weights near [2.0, -3.0] and bias near 1.0.
zero_grad() sets .grad to None, not zeros.

Validate a loop on a problem whose answer you already know. A falling loss is not proof.