Unit 05.00: What a vector store does and does not promise

A vector store returns the k nearest vectors. That is the entire promise, and everything surprising about retrieval follows from what it leaves out.

Always k results, relevant or not

There is no empty result in a nearest-neighbour search. Query for something absent from the corpus and you still get k documents back.

The code queries a three-document store for an office address.

from langchain_core.documents import Document
from langchain_core.embeddings import DeterministicFakeEmbedding
from langchain_core.vectorstores import InMemoryVectorStore

DOCS = [
    Document("Refunds are allowed within 7 days.", metadata={"section": "refunds"}),
    Document("Exchanges are allowed within 30 days.", metadata={"section": "exchanges"}),
    Document("Support replies within one business day.", metadata={"section": "support"}),
]
store = InMemoryVectorStore.from_documents(DOCS, DeterministicFakeEmbedding(size=64))

hits = store.similarity_search("office address", k=3)
print("query with nothing relevant in the corpus:")
for h in hits:
    print(f"   {h.page_content}")

print(f"\nreturned {len(hits)} results anyway")

# That is the promise and its limit. A vector store returns the k nearest
# vectors -- always. It does not promise any of them is relevant, and there is
# no empty result unless you add a score threshold yourself.

Three results, none of them about addresses. The store did exactly what it promises: it found the nearest vectors, and nearest is relative to what exists rather than to what would be good.

So an empty result - the thing you need in order to refuse - has to be constructed. That means scores and a threshold, which is what Module 6 builds. similarity_search alone cannot give it to you.

The mistake this prevents

The mistake is treating a returned document as evidence of relevance. It is evidence that the corpus contained something, ranked. A system with no threshold answers every question from whatever happened to be nearest.

Takeaway

A vector store always returns k results. Refusal requires scores and a threshold you set from your own distribution, not from the search returning nothing.

Mark complete