Unit 09.00: An embedding is a direction, not a description

An image embedding is a direction. Individual numbers in it mean nothing.

Only comparisons are interpretable

Three images embedded, with cosine similarities between them.

The code prints the vector and the comparisons.

import numpy as np

rng = np.random.default_rng(0)
vectors = {name: v / np.linalg.norm(v) for name, v in
           {"carton_a": rng.normal(size=8), "carton_b": None,
            "pallet": rng.normal(size=8)}.items() if v is not None}
vectors["carton_b"] = (vectors["carton_a"] + rng.normal(0, 0.15, 8))
vectors["carton_b"] /= np.linalg.norm(vectors["carton_b"])

def cosine(a, b):
    return float(a @ b)

for name, v in vectors.items():
    print(f"carton_a vs {name:10}: {cosine(vectors['carton_a'], v):+.3f}")

print(f"\nthe vector itself: {np.round(vectors['carton_a'], 2)}")
print("no dimension means 'brown' or 'cardboard' -- only the angles mean anything")

# An embedding is a direction. Individual numbers are not interpretable; only
# comparisons between vectors are, which is why every use of an embedding is a
# comparison.

The vector itself is uninterpretable - no dimension corresponds to 'brown' or 'cardboard'. Only the angle between two vectors carries meaning, which is why every practical use of an embedding is a comparison.

Normalising to unit length once, at index time, turns the comparison into a dot product and makes search a single matrix multiply.

The mistake this prevents

The mistake is trying to interpret individual dimensions, or to compare raw distances across differently-normalised vectors. Normalise once and compare only angles.

Takeaway

An embedding is a direction; only comparisons between directions mean anything. Normalise at index time and compare with a dot product.

Mark complete