Unit 10.03: Image or text embedding exploration

A two-dimensional scatter of a high-dimensional space is a hypothesis generator, not evidence. Check the structure numerically.

Measuring what a plot only suggests

Three clusters are planted deliberately, so the correct structure is known in advance and any claim about it can be verified.

Rather than eyeballing a projection, ask a question with a numeric answer: does a point's nearest neighbour share its label?

import torch
import torch.nn.functional as F

torch.manual_seed(0)
# Three clusters planted deliberately, so structure is verifiable.
centres = torch.tensor([[2.0, 2.0], [-2.0, 2.0], [0.0, -2.0]])
labels = torch.arange(3).repeat_interleave(40)
points = centres[labels] + torch.randn(120, 2) * 0.35

# Does the geometry match the labels? Check, do not eyeball a scatter plot.
normalised = F.normalize(points, dim=1)
sims = normalised @ normalised.T
sims.fill_diagonal_(-2.0)
nearest = sims.argmax(dim=1)
agreement = (labels[nearest] == labels).float().mean().item()
print(f"nearest neighbour shares the label: {agreement:.3f} of the time")

# Distance within a cluster versus between clusters.
same = torch.cdist(points[labels == 0], points[labels == 0]).mean().item()
across = torch.cdist(points[labels == 0], points[labels == 1]).mean().item()
print(f"mean distance within cluster 0 : {same:.3f}")
print(f"mean distance to cluster 1     : {across:.3f}")
print("separated:", across > same * 2)

# A 2-D projection of a 512-D space discards most of the variance. Use it to
# form hypotheses, then confirm them with a number like the one above.

The neighbour-agreement rate quantifies what a plot only implies. Comparing mean within-cluster distance against mean between-cluster distance gives a second, independent check.

Both take one line and neither depends on a projection that discarded most of the variance.

The mistake this prevents

Concluding from a t-SNE or UMAP plot that the embedding is good. Those projections optimise local structure for visual clarity and can manufacture clusters that are not in the original space.

Takeaway

Use projections to form hypotheses, then confirm them with a number computed in the full space.

Mark complete