Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 06.01: Not in the documents versus not retrieved

Two very different failures produce the same output. Telling them apart is the difference between fixing your corpus and fixing your retriever.

The same empty result, two causes

Either the corpus genuinely lacks the answer, or it has it and retrieval missed it. The user sees one message in both cases. You need to see two.

The example below runs three queries against a one-chunk corpus, annotated with which case each one is.

What is the refund window?       -> [('c1', 0.3333333333333333)]   (in the corpus, should retrieve)
i want my money back             -> NOTHING   (in the corpus, wording missed it)
What is the office address?      -> NOTHING   (genuinely not in the corpus)

The middle case is the dangerous one: "i want my money back" retrieves nothing from a corpus that contains the refund policy. The user is told the documents do not cover something they do, and - because the message is confident and specific - they stop asking.

The third case is the honest one. There is no address in the corpus, the refusal is correct, and the fix is to add a document rather than to change any code.

The mistake this prevents

The mistake is not logging empty results. Without that log, retrieval misses are invisible: nobody complains about an answer they did not get, and the metrics show a healthy refusal rate. The log of queries that returned nothing is your backlog of retrieval bugs and content gaps, and it costs almost nothing to keep.

Takeaway

Log every query that retrieves nothing, then review the log. Some entries are missing documents and some are retrieval failures; only reading them tells you which, and only one of the two is fixed by writing more content.