Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 03.04: Reading a retrieval result like an analyst

When an answer is wrong, the first question is whether retrieval found the right chunk. Answering that takes thirty seconds if you inspect the result properly and an afternoon if you only read the answer.

Three questions to ask before the model sees anything

How large is the gap to the next hit? Do the top hits agree with each other? How old are they? Those three questions can be answered mechanically, from fields you already have, before a single token is generated.

Below is a retrieval result and works through all three.

RETRIEVAL RESULT
  id    score   updated       text
  c1    0.91    2026-06-01    Refunds within 7 days.
  c2    0.89    2024-02-11    Refunds within 30 days.
  c3    0.42    2026-05-02    Shipping takes 3 days.

score gap, 1st to 2nd:  0.02   -- effectively tied
do the top two agree?   no     -- 7 days against 30 days
is the loser stale?     yes    -- 2024

Three questions answered before the model sees anything, from fields you
already have. A close pair that contradicts itself is the case to escalate.

The top two chunks are separated by 0.02 - effectively tied - and they state different refund windows, and the second is from 2024. Every one of those is visible without a model, and together they describe a case that should never have been answered automatically.

Had the ranking come out the other way, the assistant would have confidently reported 30 days from a stale document, with a real citation. The answer would have looked identical in every respect except the number.

The mistake this prevents

The mistake is debugging RAG by rewriting the prompt. The prompt is one stage of six, and it is the only one most people can see, so it absorbs all the blame. Inspect the retrieval result first: a large share of bad answers never had the right chunk to work from, or had it tied with a contradiction.

Takeaway

Log the retrieved ids, their scores and their dates on every query, and read them before reading the answer. A close score gap between contradictory chunks is the signature of a case that belongs with a human, not in an automatic reply.