Unit 10.01: Similarity, distance, and nearest-neighbour search basics

Cosine and Euclidean answer different questions. On unnormalised vectors they can rank the same items differently.

Direction versus position

Cosine similarity measures the angle between vectors and ignores their length. Euclidean distance measures how far apart they are, so length matters.

Take two vectors pointing the same way, one twice as long as the other:

import torch
import torch.nn.functional as F

torch.manual_seed(0)
vectors = torch.tensor([
    [1.0, 0.0],      # a
    [2.0, 0.0],      # b -- same direction as a, twice the length
    [0.0, 1.0],      # c
])
names = ["a", "b", "c"]

# Cosine cares about direction; Euclidean cares about position.
print("cosine a-b   :", round(F.cosine_similarity(vectors[0], vectors[1], dim=0).item(), 3))
print("euclidean a-b:", round(torch.dist(vectors[0], vectors[1]).item(), 3))
print("-> identical direction, but distance 1.0. The two metrics disagree.")

# Normalise and they agree, which is why embeddings are usually normalised.
unit = F.normalize(vectors, dim=1)
print("\nafter normalising, euclidean a-b:", round(torch.dist(unit[0], unit[1]).item(), 3))

# Brute-force nearest neighbour search.
query = torch.tensor([0.9, 0.1])
sims = F.cosine_similarity(query.unsqueeze(0), vectors, dim=1)
order = sims.argsort(descending=True)
print("\nnearest to the query:")
for i in order:
    print(f"  {names[i]}  cosine {sims[i]:.3f}")
print("\nExact search is O(n) per query. Approximate indexes trade a little")
print("recall for speed once n gets large -- measure that recall, do not assume it.")

Cosine says they are identical — similarity 1.0. Euclidean says they are a full unit apart. Both are correct; they measure different things.

Normalising to unit length makes the two metrics agree, which is why embedding pipelines almost always normalise before storing or comparing.

The mistake this prevents

Mixing metrics between indexing and querying — building an index under cosine and querying with Euclidean, or normalising at write time but not at read time. Results are subtly wrong rather than obviously broken.

Takeaway

Normalise, then pick one metric and use it everywhere. Measure approximate-search recall rather than assuming it.

Mark complete