Module 09 Summary

The idea this module was built around

One block: attention, add and norm, feed-forward, add and norm. Everything else is repetition and scale.

What you can now do

• Describe a transformer block's four components and why residuals make depth trainable
• Carry a mask from tokenisation through to pooling
• Use a frozen encoder as a feature extractor and cache the vectors
• Fine-tune with discriminative learning rates

The trap this module removed

Reporting a single overall accuracy for a model applied to identifiable groups. The number is true and it conceals exactly what a reviewer needs to know.

Figures worth remembering

Fine-tune the encoder around 1e-5 against a head at 1e-2. Frozen extraction trains zero encoder parameters.

Before you move on

Take a text model you have evaluated and recompute its metric per slice. Define the slices first.

Mark complete