Module 08 Summary

The idea this module was built around

An embedding is a learnable lookup table. Its geometry comes from the task it was trained on and nowhere else — an untrained table's similarities are noise.

What you can now do

• Explain what padding, truncation and masking each cost
• Say precisely what bag-of-words discards
• Describe how an RNN carries state, and why that limits it
• Compute attention weights and read what a token attended to

The trap this module removed

Mean-pooling without a mask. Padding positions get averaged into the representation, so short sequences are diluted by however much padding they happened to need.

Figures worth remembering

Attention weights sum to 1 per row. Bag-of-words sees identical vectors for "good service" and "service good".

Before you move on

Measure bag-of-words on your actual task before assuming you need a sequence model. It is often within a few points.

Mark complete