Module 10 Activity

Scenario

You are about to use an embedding for retrieval. Establish that it is useful and that it does not leak.

Task

1. Encode a dataset once and fit two different downstream tasks on the cached vectors.
2. Compare cosine and Euclidean on vectors of different lengths, then normalise and compare again.
3. Train autoencoders at several bottleneck widths and find where reconstruction error collapses.
4. Plant known clusters and verify the structure numerically rather than from a plot.
5. Probe the embeddings for a sensitive attribute, and probe the raw content for comparison.
6. Report downstream accuracy and leakage together.

Deliverable

An exploration report with downstream accuracy, raw-feature accuracy, attribute recoverability, and neighbour agreement.

Check your work

• Two vectors in the same direction have cosine 1.0 and non-zero Euclidean distance. Normalising

reconciles them.

• Reconstruction error collapses at the data's true dimensionality and barely improves above it.
• If an attribute is recoverable well above its base rate, it is influencing retrieval.

The expectation to state first

An untrained encoder is a random projection. It should not beat raw features. If it does, the evaluation is wrong.

Mark complete