Module 08 Activity

Scenario

Decide whether a task needs a sequence model or whether counting words is sufficient.

Task

1. Show that an untrained embedding's similarities are meaningless, then hand-place vectors to show what

a trained table looks like.

1. Pad and truncate a batch of sentences. Record what was lost and build the mask.
2. Build bag-of-words features and identify which rows become identical.
3. Train an RNN on data where order is the only signal.
4. Compute attention weights by hand and confirm each row sums to 1.
5. Compare bag-of-words against a sequence model on the same split.

Deliverable

A comparison note: both scores, the majority baseline, the mechanism explaining the gap, and an honest limitation.

Check your work

• Summed inputs are identical for both classes in the order-only task, so counting cannot beat chance.
• Attention rows each sum to 1.0. "it" attends most strongly to "cat".
• Truncation to 5 tokens discards "at the gate" permanently.

The limitation to state

If you built the task to favour sequence models, say so. It proves capability, not superiority.

Mark complete