Unit 09.04: Where similarity search misleads you

Five ways similarity search misleads, and one of them produces no symptom at all.

Absent objects, dominant attributes, scale, model mismatch, duplicates

Each situation with what you actually get back.

The code lists them.

FAILURES = [
    ("the corpus does not contain the object",
     "returns k neighbours anyway, all irrelevant"),
    ("one visual attribute dominates",
     "matches on background or lighting, not the object"),
    ("the query is a crop, the index is full frames",
     "scale mismatch; scores are low and rankings arbitrary"),
    ("the index was embedded with a different model",
     "vectors are not comparable; results are meaningless, not empty"),
    ("near-duplicates fill the top-k",
     "five results, one distinct image"),
]
print(f"{'situation':46} what you get")
for situation, result in FAILURES:
    print(f"{situation:46} {result}")

print("""
The fourth is the one that produces no symptom at all. Vectors from two
different models occupy different spaces, so the search runs, returns results,
and ranks them by nothing. Record which model produced an index.
""")

The fourth is the one to guard against structurally. Vectors from two different embedding models occupy different spaces, so a query embedded with model B against an index built with model A returns results, ranks them, and the ranking means nothing.

There is no error and no obviously wrong output - just quietly random results. Record which model produced an index, and refuse to search it with another.

The mistake this prevents

The mistake is rebuilding an index with an upgraded embedding model and leaving old cached vectors in place. The mixture is undetectable from the results and permanently degrades the search.

Takeaway

Record which model produced every index and never mix. Model mismatch produces plausible, meaningless rankings with no error anywhere.

Mark complete