Module 02 Summary
The idea this module was built around
The first dimension is the batch. Most PyTorch errors are shape errors, and most shape errors are a lost or mismatched batch dimension.
What you can now do
- Read shape, dtype and device, and restore a batch dimension with
unsqueeze(0) - Convert tabular data to tensors with an explicit dtype
- Predict what broadcasting will do before running it
- Split reproducibly and verify the splits do not overlap
The trap this module removed
An (n,) target against an (n, 1) prediction does not error. It broadcasts to (n, n), the loss trains on a matrix of pairwise differences, and the model appears to learn while optimising nonsense.
Figures worth remembering
20 items at batch size 6 gives four batches, the last holding 2. drop_last=True discards them.
Before you move on
Print the shape of both operands before any loss computation. It costs one line and catches the quietest bug in this course.