Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 05.00: Answering only from what came back

Grounded answering is a constraint, not a style. Either the answer is built from retrieved text or it is not, and the difference has to be visible in the code rather than requested in a prompt.

The branch that has to exist

An answering step faces two situations: the context supports an answer, or it does not. Both need a return path. A design with only the first path will always produce something, because producing something is the default.

The example below answers two questions from a single-chunk context - one covered, one not.

CONTEXT
  [c1] Refunds are allowed within 7 days of purchase.

QUESTION                       ANSWER                                   CITES
What is the refund window?     Refunds are allowed within 7 days.        c1
What is the shipping cost?     The available documents do not cover      (none)
                               that.

The second question has no supporting chunk, so the answer declines and names
no source. The empty citation list is machine-readable evidence that nothing
was claimed -- which is what lets a check tell a refusal from an answer.

The refund question is answered with a citation. The shipping question returns a decline with an empty citation list. Nothing about the second output is apologetic or hedged; it simply reports that the material is absent.

The empty cites list matters as much as the text. It is machine-readable evidence that no source was claimed, which means an automated check can distinguish a refusal from an answer without parsing English.

The mistake this prevents

The mistake is implementing grounding entirely in the prompt and leaving the code with a single return path. When the prompt is ignored - and it will be, on some fraction of queries - there is no structural constraint left. The if not context branch is the part that holds when the model does not.

Takeaway

Grounded answering needs an explicit decline path in the design, with fixed wording and an empty citation list. That structure is what later checks and eval sets can assert against.