Unit 06.03: Handling nothing relevant found

Refusal requires a score and a threshold, because the search itself will never come back empty.

Scored retrieval plus a threshold

similarity_search_with_score returns the distance alongside each document, which is what lets you discard everything below a bar.

The code answers a covered question and an uncovered one.

from langchain_core.documents import Document
from langchain_core.embeddings import DeterministicFakeEmbedding
from langchain_core.vectorstores import InMemoryVectorStore

REFUSAL = "The available documents do not cover that."
store = InMemoryVectorStore.from_documents(
    [Document("Refunds are allowed within 7 days.", metadata={"id": "c1"})],
    DeterministicFakeEmbedding(size=64))


def answer(question, threshold=0.75):
    """Score-aware retrieval: an empty result is a decision, not an accident."""
    scored = store.similarity_search_with_score(question, k=1)
    keep = [(d, s) for d, s in scored if s >= threshold]
    if not keep:
        best = scored[0][1] if scored else 0.0
        return {"text": REFUSAL, "cites": [], "refused": True, "best_score": best}
    doc, score = keep[0]
    return {"text": f"{doc.page_content} [{doc.metadata['id']}]",
            "cites": [doc.metadata["id"]], "refused": False, "best_score": score}


for q in ["What is the refund window?", "What is the office address?"]:
    out = answer(q)
    print(f"{q:32} refused={out['refused']!s:5} score={out['best_score']:.3f}")
    print(f"   {out['text']}")

# `similarity_search` alone always returns something. Only the scored variant
# plus a threshold gives you a path where nothing comes back -- and the
# threshold has to be set from your own score distribution, not guessed.

The refusal returns refused: True, an empty citation list, and the best score it saw. All three are useful: the flag is machine-readable, the empty list proves no source was claimed, and the score is what you use to tune the threshold later.

The threshold has to come from your own score distribution. Run the eval set, look at the scores for answerable and unanswerable questions, and put the bar where they separate - if they separate at all, which is itself worth knowing.

The mistake this prevents

The mistake is picking a threshold from a tutorial. Score scales differ by embedding model and by distance metric, so a number that works in one setup refuses everything or nothing in another. Measure yours.

Takeaway

Use scored retrieval and a threshold set from your own distribution. Log the best score on every refusal - that log is what lets you tune the threshold rather than guess it.

Mark complete